How to reduce LLM token costs: the 2026 solutions landscape
TL;DR: cutting an LLM bill comes down to a short list of techniques — use a cheaper model where the task allows, cache re-sent context, trim and retrieve instead of stuffing context, cap loops and outputs, and batch what you can — and a set of tools that help you find and apply them. The tools fall into five categories: observability, gateways/routers, eval platforms, FinOps dashboards, and code-level cost analysis. Each has a genuine fit; most stop at telling you what you spent or rerouting traffic, and only code-level analysis finds the waste in your source and proves a fix before it ships. This page lays out both halves honestly so you can pick the right combination.
Part 1 — the techniques that actually move the bill
These are the changes that reduce token spend, roughly in order of how much they tend to save:
- Cache re-sent context. If a loop or agent re-sends the same large prefix (system prompt, tool schemas, a growing transcript) every turn, you pay full price for tokens the model already saw. Caching bills those reads at roughly a tenth. Usually the single biggest lever — details and a caveat in The caching trap.
- Right-size the model. A frontier model on a simple task (classification, routing, extraction, a short label) is the most common leak after caching; the cheap tier is often 10x–100x less. See the LLM API pricing table for the gaps, and how to prove a swap holds before you ship it.
- Cap loops and outputs. A max-turns cap on agent loops and a
max_tokenson every call stop runaway spend. - Trim and retrieve instead of stuffing context. Sending whole files or full histories when a slice would do scales waste with traffic — see the hidden cost of long context windows.
- Batch independent calls. Many one-at-a-time calls that don't depend on each other can often be a single batched request.
The full taxonomy, with how to estimate each honestly, is in the complete guide to LLM cost optimization. The catch running through all of them: several are behaviour changes, so a saving is a candidate until you verify quality holds.
Part 2 — the tool categories, compared
No single tool does everything, and the honest framing is that they sit at different points in the lifecycle — from "after the spend happened" to "before it reaches production."
| Category | What it does | Best for | What it misses |
|---|---|---|---|
| Observability (Helicone, Langfuse, LangSmith, Datadog) | Logs and dashboards your spend per model/request, after calls happen | Seeing where money already went in production | Can't change the code or tell you the fix — it reports, it doesn't prevent |
| Gateways / routers (OpenRouter, Portkey, LiteLLM, Martian, RouteLLM) | Sit in the request path and reroute or cache live traffic | Centralizing keys and rerouting at runtime | Moves the lever out of your code into infra; doesn't rewrite your code or prove a cheaper model holds quality |
| Eval / comparison platforms (promptfoo, Braintrust, Vellum) | Run a prompt across models and compare cost vs quality | Deciding which model to use, if you bring your own test set | You write the eval, make the call, and do the rewrite by hand — no automatic recommendation or fix |
| FinOps / spend dashboards (Vantage, CloudZero) | Aggregate AI spend into finance views and budgets | Finance and procurement visibility across an org | Operates at the invoice level; no code-level cause or fix |
| Code-level cost analysis (erabot) | Scans your source, estimates the dollar cost per call site, generates the fix, and shadow-tests that a cheaper option holds quality | Finding and fixing the waste before it ships, with a proof | Static analysis of code, not a runtime traffic layer — pairs with observability rather than replacing it |
Each row is a real, useful product category — the point is not that four of them are bad, it's that they answer different questions. Most stop at what you spent (observability, FinOps) or reroute what you send (gateways). The gap the last row fills is reading the code where the cost decision was actually made, and proving the fix is safe. (We wrote about that specific gap in Observability tells you what you spent; it can't tell you how to spend less.)
Part 3 — which to use when
- "I'm in production and want to see where the money goes." Start with observability. It won't fix anything, but it shows you the shape of the spend.
- "I want to centralize keys and reroute traffic." A gateway/router. Useful infra; just know it doesn't touch your code or prove quality.
- "I need to pick a model for a task." An eval platform, if you have (or will write) a test set. Or read a leaderboard for a shortlist, then verify on your own data.
- "I want to find the waste in my code and fix it before it ships, with proof it holds quality." Code-level analysis. This is where erabot lives: it scans the repo (or any public one), resolves the model even when it's set in config or behind a wrapper, estimates the monthly cost per call site with assumptions marked, and the verified audit shadow-tests each fix.
In practice teams combine them: observability to watch the live bill, a gateway if they want runtime control, and code-level analysis to catch and fix the waste at the source. They're complements, not substitutes.
"Use a cheaper model" is where tools quietly differ
Most tools that suggest a cheaper model pick it the same way: off a public leaderboard score, or as an automatic swap applied for you. A leaderboard rank is a generic average across everyone's prompts — a fine way to build a shortlist, but not evidence the cheaper model holds up on your task, and an auto-swap on that basis can quietly ship a quality regression. The honest version tests the candidate against your own examples first and reports whether it actually held — a proof, not a proxy. That is the whole difference between "here is a cheaper model" and "here is a cheaper model we verified holds quality," and it is the step worth not skipping (the full method is in how to prove a model swap holds quality).
The honest bottom line
Reducing LLM costs is not one purchase — it's the five techniques above, applied where they fit, and verified so you don't trade a lower bill for worse output. Tools help you find and prove those changes faster; none of them removes the need to make the call on your own task. If you want the fastest honest starting point, scan your code: it turns "my bill is high" into a ranked list of specific call sites and dollar estimates in a few minutes.
Scan your repo (or any public repo) free with erabot — ranked findings, honest dollar estimates, assumptions marked. The verified audit then proves each fix holds quality before you ship. Start at erabot.ai.