Model Routing
Sending each request to the cheapest model that can handle it.
Directing requests to different models based on difficulty, task type or cost target — a small fast model for classification, a frontier model for hard reasoning, with an escalation path between them.
Usually the single largest cost lever available, and it improves latency too. The trick is a reliable signal for difficulty, often a classifier or a cheap first attempt with a confidence check.
The design question is the escalation rule: what makes a request hard, and what happens when the cheap model is unsure. The second half is what separates routing that works from routing that quietly degrades quality — a confidence signal, a grader, or a fallback on failure all work, and doing nothing does not.
Routing with a fallback
Most requests are simple lookups and a tenth need real reasoning.
Send everything to the small model, and escalate when it flags low confidence or fails the schema. Cost falls substantially while quality holds, because the escalation path exists — routing without one is simply trading quality for price.
- docsPricing ↗
The current per-token and per-service prices. The right kind of source for token economics — and the reason no cost figure should be hard-coded without an access date.
- blogIntroducing Claude 4 ↗
Hybrid models with near-instant and extended-thinking modes, reported with coding and agentic evaluations — a concrete picture of what "reasoning model" means in a product.