Test-time Compute

The idea that a model can spend more computation when answering — generating a longer reasoning chain, sampling several attempts, searching over options — to improve the result. It is a lever you control per request rather than something fixed at training time.

This reframed the cost model: quality is no longer purely a property of which model you called, but of how much you let it think. The trade-off is explicit and per-request, which is why routing and budgets became standard architecture.

There is a diminishing-returns curve and it is worth measuring rather than assuming. Past a point, longer reasoning chains wander, restate, or talk themselves out of a correct answer. Your eval set is the only way to find where the curve flattens for your task.

Assuming more thinking always helps. On easy tasks it adds cost and latency, and very long chains can drift away from the question.

• docsExtended thinking ↗Claude platform docs · 2026

  Configurable thinking budgets and interleaved reasoning around tool results — the API-level view of chain of thought as a resource you pay for.

• paperChain-of-thought prompting elicits reasoning in large language models ↗Wei et al. (arXiv) · 2022

  Few-shot demonstrations containing intermediate reasoning steps improved results on arithmetic, commonsense and symbolic reasoning tasks.

• paperDeepSeek-R1: incentivizing reasoning capability in LLMs via reinforcement learning ↗DeepSeek-AI (arXiv) · 2025

  Reinforcement learning on verifiable tasks elicited reasoning patterns — self-checking, backtracking — with substantial open publication of the recipe.