Input Tokens
also called Prompt tokens
What you send. Cheaper, but repeated on every request.
The tokens in the request — system prompt, history, retrieved documents, user message. Billed at the input rate, typically several times cheaper than output.
In a chat app, input tokens grow with every turn because the whole history is resent. This is why long conversations get expensive, and why prefix caching exists.
Input tokens are where the cheap wins live, because they are repeated on every request in a conversation. A system prompt that is resent on turn twenty has been paid for twenty times — which is exactly the arithmetic that makes caching and prompt discipline worth the effort.
The instruction preamble paid for a thousand times
A chat product resends a 4,000-token instruction block on every turn.
Confirm it sits at a stable prefix, then let caching handle it. The quality does not change at all; the input line on the invoice does. This is the least invasive optimisation available to most applications.
- docsPrompt caching ↗
Caching reuses the model’s key-value state for a matching prompt prefix. Explains why reordering your system prompt throws away the cache.
- docsPricing ↗
The current per-token and per-service prices. The right kind of source for token economics — and the reason no cost figure should be hard-coded without an access date.
Output Tokens · Completion tokens
What the model writes. Pricier, and the main driver of latency.
Token
The chunk of text the model actually sees — and the unit you are billed in.
Prefix Cache · Prompt caching
Reusing the model’s internal work for a prefix it has already seen.
Cache Tokens
Tokens billed at the cached rate — cheap reads, sometimes a write fee.
Cost per Task
What one unit of real work actually costs.