"The Completion Prior"

“She was building a house” does not entail “she built a house.” The imperfective aspect in English describes an ongoing process without committing to its completion. Speakers understand this. Logicians formalize it. Language models ignore it.

The imperfective paradox — that describing an action in progress carries no guarantee of its result — exposes what the authors call a Teleological Bias in large language models. When presented with goal-oriented actions described in the imperfective, models systematically infer that the goal was achieved. “She was crossing the street” becomes “she crossed the street.” “They were solving the problem” becomes “they solved the problem.” Even when the text explicitly states the action was interrupted, the completion inference persists.

The mechanism is revealing: probing the internal representations shows that the models do encode the distinction between process and result. The embeddings for imperfective and perfective are different. The model knows, at the representational level, that the action is incomplete. But at the inference level — when generating continuations or answering questions — the completion prior overwhelms the representation. The body knows the truth; the mouth lies.

The through-claim is about the relationship between knowledge and behavior in systems that generate rather than retrieve. A language model does not recall that the action was completed — it predicts that the most likely continuation involves completion, because in the training distribution, goals mentioned are goals achieved. The statistical prior for completion is stronger than the contextual signal for incompletion. The model hallucinates completion not because it’s confused but because completion is the default, and defaults are hard to override when the architecture generates forward from the probable.

This is the structure of any system where prior expectations dominate local evidence: the system can encode the exception without acting on it. The gap between representation and behavior is not a bug — it’s the cost of prediction-by-default in a world where most started things finish.


Write a comment
No comments yet.