Cache Tokens
Tokens billed at the cached rate — cheap reads, sometimes a write fee.
The portion of a request served from a prefix cache. Cache reads are billed well below normal input rates; cache writes may cost slightly more than a normal token.
Caching is only worth it if the prefix actually repeats. A cache you write on every request but never reuse is a small net loss.
The accounting detail that surprises people: cache writes can cost slightly more than ordinary input tokens, so caching pays off through repetition and not automatically. A prefix used once is a small loss; used a thousand times it is the cheapest spend in the system.
Reading a cost line item properly
An invoice shows a much lower input total than the volume implies.
That is the cache doing its job: the majority of input tokens were billed at the cached read rate. Comparing cost per task before and after caching is the clean way to prove the win, rather than eyeballing list prices.
- docsPrompt caching ↗
Caching reuses the model’s key-value state for a matching prompt prefix. Explains why reordering your system prompt throws away the cache.
- docsContext caching overview ↗
Caching a large, reused prefix so you stop paying full price to resend it — and the cache-lifetime and prefix-stability conditions that decide whether it hits.