Token
The chunk of text the model actually sees — and the unit you are billed in.
A token is a piece of text, often a word or part of a word. Models read and write tokens, not characters. Roughly 4 characters or 0.75 words per token in English.
Token counts drive three things at once: how much fits in context, how much each request costs, and how long the response takes. When someone says "one million tokens", they mean roughly 750,000 words.
Because tokens are the billing unit, they are also the unit of architectural decisions. Every context-management technique in this compendium — summarising, retrieving, caching, pointing at a file instead of pasting it — is a way of buying the same outcome with fewer tokens.
The estimate that survives a meeting
Someone asks whether a document will fit in the window.
Divide characters by four for English prose. A 200-page PDF is roughly 100,000 tokens, which is a fact you can establish in ten seconds without opening a tokeniser — enough to know whether the plan is plausible before anyone builds it.
- docsGet token count ↗
How text and multimodal input are tokenised and counted, with usage reporting. Concrete counts are model-specific; the concept is not.
- paperLanguage models are few-shot learners ↗
Introduces GPT-3 at 175B parameters and shows that scale alone produced in-context learning: performance from examples in the prompt, with no gradient updates.
Tokenizer
The program that chops text into tokens before the model sees it.
Context Window
The hard ceiling on how much the model can see at once.
Input Tokens · Prompt tokens
What you send. Cheaper, but repeated on every request.
Output Tokens · Completion tokens
What the model writes. Pricier, and the main driver of latency.
Next-token Prediction
The single objective: guess what comes next, one token at a time.