Is the cheaper model actually cheaper?
A cheaper model that fails twice costs more than an expensive one that works first time. The catch is that the cost of failure never lands on the model that failed: retries inflate call volume, escalations appear under the premium model's line, and a bad answer someone has to fix appears in salaries. Put your own numbers in and see which way it actually falls.
Workload
Candidate model — the cheaper one you want to route to
Premium model — the fallback, and the comparison
Route to candidate, escalate on failure
Retry the cheap model, fall back to premium.
— per successful outputPremium model only
Every task to the expensive model.
— per successful outputproductive spend · retries, escalations and rework — the part a per-token chart hides
—
The number that misleads: the candidate looks — — — against — per call. That is the figure on every pricing page and most dashboards, and it is the one that has nothing to say about whether the work got done.
How the maths works
Each attempt is treated as an independent trial at the model's success rate. Tasks go to the candidate model and are retried up to your attempt limit; anything still failing escalates once to the premium model. Spend is summed across every attempt — successful or not — plus the cost of any task that ends without an acceptable output, then divided by the outputs that actually succeeded.
Two things to be straight about. Treating retries as independent flatters the cheap model: in reality a prompt that fails once is more likely to fail again, so the candidate's true cost is at least what this shows. And the cost of an escaped failure is the term that most often flips the verdict — it is also the one nobody has in their cost tooling, because it lands in salaries rather than on the AI line of the bill.
This is a decision aid, not a forecast. The inputs that matter most — your two success rates — should come from a real evaluation on a sample of your own prompts. How to define and instrument a success bar is the companion to this tool.
Once you know the shape
Model routing is the largest saving available in most AI workloads, and the only one gated on a quality test rather than a configuration change. Before running that test, it is worth knowing what you are already paying for nothing: a deployment serving zero requests, provisioned throughput billing flat against traffic you do not have, a prompt cache that never warms.
The CloudFinOpsKit FinOps Agent reads that from your real billed cost across Azure OpenAI, Amazon Bedrock and Vertex AI — no agent, no proxy, no code change. See what it measures and what a finding looks like.
Related: cost per successful output, explained · the token efficiency framework · FinOps for AI: the complete framework · cloud cost savings calculator