Hybrid LLM Routing Cost Calculator

You keep a frontier model for planning and review, and send the rest of the work to a local box or a cheap API. Enter the month's tokens, the share you route, and how often the cheaper model fails. The calculator prices the month three ways: all on the frontier, routed to a cheap API, and routed to your own box.

How it works

all-frontier   = API bill for every token (uncached input, cached input, output)
frontier part  = bill for the work it keeps
               + bill for the routed work x fail share   (done again)
               + routed output x review share x uncached input price
cheap API part = API bill for the routed tokens at the executor's prices
box part       = (price paid - sale price x (1 - selling fee)) / months
               + power for the hours the routed work takes
hybrid bill    = frontier part + cheap API part, or frontier part + box part
break-even fail share = the fail share at which hybrid bill = all-frontier

Worked example: 200M input and 4M output tokens a month on Claude Opus 5.5

80% frontier cache hits, 10% of routed work fails, the frontier reviews all routed output. Cheap API: Qwen3.6 27B (OpenRouter, cheapest host). Box: RTX 3090 24GB (used) at $1,649, kept 24 months.

Work routedFrontier tokens movedBill, cheap API executorBill, bought boxBill, box you own
0%0%$312.00$321.34 to $345.09$312.00
30%26%$240.36$243.89 to $267.63$234.54
50%44%$192.60$192.25 to $215.99$182.90
70%62%$144.84$140.61 to $164.35$131.26
90%79%$97.08$88.97 to $112.71$79.62

The 0% row is the all-frontier bill, $312.00. A bought box that does no work still costs its monthly hardware share. Routing 70% of the work cuts the bill by 54%, not 70%.

What the fail share does

Routed work that failsDone again on the frontierBill, cheap API executorBill, bought boxSaved vs all-frontier
0%$0.00$123.00$118.77 to $142.5161%
10%$21.84$144.84$140.61 to $164.3554%
25%$54.60$177.60$173.37 to $197.1143%
50%$109.20$232.20$227.97 to $251.7126%

At 70% routed, the hybrid month costs the same as the all-frontier month when 87% of the routed work fails (cheap API executor). The token bill forgives a weak executor for a long time. Your review time does not.

Viewer claims, run through the math

These are comments on CloudYeti local-AI videos. The figures in quotes are the viewers' own. The dollar figures are this calculator's, at the default month and the assumptions above.

Questions

How much does hybrid LLM routing save?

Less than the share of work you route. In the default example (200M input and 4M output tokens a month on Claude Opus 5.5, 80% cache hits), routing 70% of the work to a cheaper executor moves 62% of the frontier tokens and cuts the $312.00 bill to $144.84 with a cheap API executor, a 54% saving. The frontier still pays for the work it keeps, for the 10% of routed work that fails and is done again, and for reading the executor's output back.

Is LLM routing cheaper than using one frontier model for everything?

On the token bill, yes, as long as the cheaper model's fail share stays under the break-even. In the default example the break-even is 87%: the routed work would have to fail that often before the all-frontier month is cheaper. The bill does not include your time checking the cheaper model's work, and that is usually the larger cost.

Should I route to a local model or to a cheap API?

On cost alone they are close when the box is small and the volume is high, and the cheap API wins at low volume. At the default month a used RTX 3090 at $1,649 makes the hybrid bill $140.61 to $164.35, against $144.84 with Qwen3.6 27B on a cheap host. At one tenth of that volume the cheap API route costs $14.48 and the bought box $22.47 to $46.22. A box you already own costs only its power.

Does a local model pay for itself if I keep a frontier model for planning?

It can, when the all-frontier bill is large. At the default month the saving pays for a $1,649 used RTX 3090 in 9.1 months, before any resale. To pay it back in 90 days at the same mix, the all-frontier bill has to be about $949 a month.

Does a $20 or $200 AI subscription change the routing math?

Yes. A subscription is a flat fee with usage limits, so routing work away from it saves no dollars. It buys room under the limit. Claude Pro is $20 a month, Max $100 and Ultra $200 (claude.com/pricing, read 2026-11-20). This calculator prices API tokens. Use it when you pay per token, or when you would have to upgrade a plan to fit the work.

Related