Attention Degradation
also called Lost in the middle
Recall drops for content buried in a large window.
The observed decline in a model’s ability to use information as the window grows, and especially for material in the middle of a long context — notices hold up better at the very start and very end.
Even models that pass needle-in-a-haystack tests show degradation on tasks that require reasoning across many pieces at once. Fitting it in the window is not the same as using it.
The measured shape is a gradient, not a cliff: material at the start and end of a long context is used more reliably than material in the middle. That is directly actionable — put instructions last and the most important retrieved evidence first, rather than trusting whatever order the retriever produced.
Ordering the context deliberately
Two documents are supplied and only the second one matters.
Put it first. The difference is measurable and costs nothing beyond deciding the order — which is why context assembly deserves to be explicit code rather than an accident of how the pieces were concatenated.
- paperLost in the middle: how language models use long contexts ↗
Measured U-shaped performance: relevant information in the middle of a long context is used less reliably than at the beginning or end. Order your context deliberately.
- docsContext windows ↗
What actually consumes the window — input, tools, thinking and output — plus context rot and compaction. Authoritative for this product, not for models generally.
Attention · Self-attention
The mechanism that lets each token weigh every other token in the window.
Attention Budget
The finite share of the model’s focus each token gets.
Smart Zone
The window size where the model is still sharp.
Context Rot
A long session slowly getting worse.
Context Window
The hard ceiling on how much the model can see at once.