Context Window
The hard ceiling on how much the model can see at once.
The maximum number of tokens a model can accept in a single request — input plus output combined. Modern models range from 8k to millions of tokens.
A large window is not a substitute for good assembly. Filling it with everything raises cost, adds latency, and — past a certain point — lowers accuracy, because attention gets diluted.
The number on the spec sheet is a ceiling, not a working budget. Two forces eat into it in practice: the model reasons less reliably as the window fills, and output must be reserved from the same ceiling. The effective working window is materially smaller than the advertised one.
The window that was full of its own transcript
A long agent run starts producing incoherent edits after an hour.
The window is full of successful tool output, failed attempts and corrections — and the model is now reasoning about its own noise. Compaction plus a written handoff onto a clean session restores quality in one step.
Reading the fine print on a long-context claim
A provider advertises a million-token window.
Ask how it was evaluated. Needle-in-a-haystack suits single-fact recall, while position-sensitive studies show mid-context material is used least reliably. The window tells you what fits, not what will be reasoned about well.
Treating a big window as free. You pay per token in context on every request.
- docsContext windows ↗
What actually consumes the window — input, tools, thinking and output — plus context rot and compaction. Authoritative for this product, not for models generally.
- paperLost in the middle: how language models use long contexts ↗
Measured U-shaped performance: relevant information in the middle of a long context is used less reliably than at the beginning or end. Order your context deliberately.
- blogIntroducing the next generation of Claude ↗
A first-party example of a model family positioned by capability, speed and cost, with vision input and a needle-in-a-haystack long-context evaluation.
Context
Everything the model can currently see.
Token
The chunk of text the model actually sees — and the unit you are billed in.
Attention Budget
The finite share of the model’s focus each token gets.
Context Rot
A long session slowly getting worse.
RAG · Retrieval-Augmented Generation
Look things up, then put what you found in the prompt.