How the cascade savings are calculated
The calculator compares three ways of serving the same monthly workload. Each request has an average input size and output size, so a model’s cost per request is:
cost per request = (input tokens ÷ 1,000,000) × input price + (output tokens ÷ 1,000,000) × output price
| Strategy | Monthly cost formula |
|---|---|
| Baseline (strong only) | N × strong |
| Cascade | N × cheap × (1 + gate overhead) + N × (1 − p) × strong |
| Upfront router | N × p × cheap + N × (1 − p) × strong |
where N is requests per month, p is the fraction of requests whose cheap-model answer passes the quality gate, and strong / cheap are the per-request costs. In a cascade the cheap model runs on every request, so the rejected share is paid twice. The break-even acceptance rate is cheap ÷ strong: below it, cascading costs more than just using the strong model.
Worked example
100,000 requests a month, 1,500 input and 500 output tokens each, Claude Opus 5 as the strong model ($5 / $25 per 1M) and Claude Haiku 4.5 as the cheap model ($1 / $5 per 1M), 70% acceptance.
| Step | Value |
|---|---|
| Opus 5 per request | 0.0015 × 5 + 0.0005 × 25 = $0.0200 |
| Haiku 4.5 per request | 0.0015 × 1 + 0.0005 × 5 = $0.0040 |
| Baseline | 100,000 × $0.02 = $2,000 |
| Cascade | 100,000 × $0.004 + 30,000 × $0.02 = $400 + $600 = $1,000 (50% saved) |
| Upfront router | 70,000 × $0.004 + 30,000 × $0.02 = $880 (56% saved) |
| Break-even acceptance | 0.004 ÷ 0.02 = 20% |
The 50% figure lands inside the 36% to 67% range GitHub reported for HydraFusion. That is what you would expect from a price ratio around 1:5 and an acceptance rate somewhere between 55% and 85%, so the GitHub numbers are plausible without needing anything exotic in the routing.
2026 API prices used by this calculator
Standard tier, US dollars per 1 million tokens, text input and output. Read from each provider’s official pricing page on 21 September 2026.
| Model | Input | Output | Note |
|---|---|---|---|
| OpenAI GPT-6 Astra | $10.00 | $50.00 | |
| OpenAI GPT-5.6 Sol | $4.00 | $20.00 | Promotional; list $5 / $30, promo through at least 21 Nov 2026 |
| OpenAI GPT-5.6 Terra | $2.00 | $12.00 | |
| OpenAI GPT-5.6 Luna | $0.20 | $1.20 | |
| Anthropic Claude Fable 5.1 | $10.00 | $50.00 | |
| Anthropic Claude Opus 5 | $5.00 | $25.00 | |
| Anthropic Claude Sonnet 5 | $2.00 | $10.00 | |
| Anthropic Claude Haiku 4.5 | $1.00 | $5.00 | |
| Google Gemini 3.1 Pro Preview | $2.00 | $12.00 | Prompts up to 200k tokens; $4 / $18 above |
| Google Gemini 3.8 Flash | $0.75 | $3.75 | Through 31 Dec 2026; $1.50 / $7.50 from 1 Jan 2027 |
| Google Gemini 3.5 Flash-Lite | $0.30 | $2.50 | |
| DeepSeek V4 Pro | $1.32 | $3.96 | Peak rate, cache miss; off-peak is half |
| DeepSeek Flash (V4.1) | $0.30 | $1.20 | Peak rate, cache miss; off-peak is half |
Sources:OpenAI API pricing,Anthropic pricing (API tab),Gemini Developer API pricing,DeepSeek models and pricing. Providers change prices without notice. Check the source page before you commit a budget to any figure here.
The acceptance rate is a guess until you measure it
Everything above hinges on one input you cannot look up anywhere: how often the cheap model's answer is good enough. Extraction, classification, and short factual answers tend to clear a gate 80% to 95% of the time on a small model. Multi-file code changes and long reasoning chains might clear 30% to 50%, sometimes less. The gate moves the number too. Loosen it and acceptance goes up while bad answers slip through; tighten it and you throw away good cheap answers and drift back toward the baseline cost. Run a few hundred real requests through the cheap model, grade them, and use that rate. A projection built on a guessed 70% is worth very little.
Two costs are missing from the token math. Latency is the obvious one: an escalated request waits for two model calls in series, so tail latency roughly doubles for the rejected share. The gate itself is the one people forget. A judge model that reads the input plus the draft usually costs 100% to 150% of the cheap call, which is why the overhead field exists. If your gate is a test run or a schema check, the overhead is close to zero, and that is the setup where cascading really pays.
Reference values
| Reference value | Figure | Source |
|---|---|---|
| HydraFusion cost reduction vs Claude Opus 5 | 36% (DeepSWE) to 67% (TerminalBench 2.1) | GitHub blog, Sep 2026 |
| Batch API discount | 50% on input and output | OpenAI, Anthropic, Google pricing pages |
| Break-even acceptance rate | cheap cost ÷ strong cost | Derived above |
| Rough token size | 1 token ≈ 4 English characters | OpenAI tokenizer docs |
Frequently Asked Questions
What is cascade routing for LLM APIs?
You send every request to a cheap model first. Something checks the answer: a schema check, a test run, a small judge model, or a confidence score. If the check fails, the same request goes to a stronger, pricier model. GitHub's HydraFusion preview in Copilot CLI works this way, and GitHub reports 36% to 67% lower estimated token cost than running Claude Opus 5 alone on its coding benchmarks. Those are their numbers on their benchmarks, so treat them as an upper bound for your own workload until you measure it.
Why does the cascade cost more than the upfront-router estimate?
Because in a cascade you pay twice for every request that fails the gate: once for the cheap attempt that got thrown away, once for the strong model that fixed it. An upfront router picks the model before the call, so it pays once per request. The catch is that the router has to guess right without seeing an answer, and a wrong guess sends an easy task to the expensive model or a hard task to the cheap one. Both numbers are shown so you can see what the rejected attempts are actually costing.
What acceptance rate do I need for cascade routing to save money?
Divide the cheap model's cost per request by the strong model's. If the cheap model costs 20% as much, you break even at 20% acceptance and save money above it. In practice the number is low enough that almost any sensible model pair clears it; the real question is how far above break-even you land, because that decides whether the savings are 10% or 60%.
Where do the prices come from?
From each provider's own pricing page: OpenAI (openai.com/api/pricing), Anthropic (claude.com/pricing, API tab), Google (ai.google.dev/gemini-api/docs/pricing), and DeepSeek (api-docs.deepseek.com). Standard tier, per 1 million tokens, US dollars, read on the date shown in the price table below. Promotional prices are labeled with their expiry. If your contract rate differs, pick Custom and type your own numbers.
Does this include prompt caching, batch discounts, or thinking tokens?
No. Batch tiers cut prices by 50% at OpenAI, Anthropic, and Google, and cached input tokens cost a fraction of the standard input rate. Thinking tokens count as output tokens at all four providers, so they belong in your average output figure. To model caching or batch, choose Custom and enter your blended effective rates.
How do I estimate my average tokens per request?
Your provider's usage dashboard has the totals. Divide input tokens by request count, then do the same for output. If you have no data yet, 1 token is roughly 4 characters of English. A 1,500-token system prompt plus a short user message and a 500-token answer is a normal chat request. Agentic coding turns are a different animal, often 10 to 50 times larger, which is why HydraFusion-style savings look so large in coding tools.