erabot.ai
ProductPricingCase studiesEvalBlogDocs
Book a demo

Tool-schema bloat: the 70-97% token cut hiding in your function definitions

TL;DR: When you give a model tools or functions, their full JSON schema and descriptions ride along as input tokens on every request — and an agent with many tools re-sends all of them on every turn. Research on rewriting verbose tool documentation into a concise "tool instruction" cut tokens per tool by 70.43% on one benchmark and 97.35% on another (EASYTOOL, NAACL 2025). That number is a cut to tool documentation, not to your whole bill — the dollar impact depends on how often those schemas get re-sent. Here is how to find the bloat in your own code and trim it without breaking tool-calling.

The part of function calling nobody looks at

When you ask a model to call functions, you hand it a list of tool definitions: a name, a description, and a JSON schema of parameters — each parameter often carrying its own prose description. The model never sees your code; it sees that text. Which means every character of it is input tokens, billed on the request that carries it.

For a single call with two or three small tools, this is noise. The problem is agents. An agent loops: it reads the tools, picks one, gets a result, and loops again to decide the next step — and on each turn the entire tool list goes back up the wire. Ten verbose tools re-sent across a twenty-turn trajectory is the same schema paid for twenty times. This is the function-calling cousin of the re-sent-context leak we covered in the complete cost-optimization guide: the cost is not in any one call, it is in the repetition.

Caching helps, but it does not make the problem disappear. Provider prompt caching discounts a re-sent prefix; it does not delete it, the discount only lands if the tool block sits in the stable prefix and the cache actually hits, and on the turn the schemas are first sent you pay full price regardless. Trimming the schema cuts the tokens whether or not a cache ever fires — which is why it is worth doing at the source.

The evidence: 70-97%, with an honest asterisk

The sharpest published number comes from EASYTOOL (NAACL 2025). The authors took verbose, redundant tool documentation and rewrote it into a concise, unified "tool instruction" — keeping what the model needs to call the tool correctly and dropping the rest. The result:

  • ToolBench: average tokens per tool fell 70.43%, from 2,530 to 748.
  • RestBench: average tokens per tool fell 97.35%, from 3,881 to 103.

Source: EASYTOOL, aclanthology.org/2025.naacl-long.44.

Two caveats matter, and skipping either one would be dishonest:

First, EASYTOOL is an automated LLM-based fix, not a static detector. The paper used a model to rewrite the documentation; it did not build a scanner that flags bloated schemas in source. Treating its result as evidence that a static detector can find the same waste is a reasonable inference — bloated tool docs are visible in code without running anything — but it is an inference on our part, not a claim the paper makes. We will not dress it up as more than that.

Second, 70-97% is a cut to tool documentation tokens, not to your total bill. Those percentages describe how much smaller the tool-definition text got. What that saves you in dollars depends entirely on how large a share of your input tokens the tool block actually is, and on how many times it gets re-sent. If your tools are a sliver of each prompt, trimming them 97% is a rounding error; if you run a many-tool agent over long trajectories, it can be a real line item. Anyone who converts "97% smaller schemas" into "97% off your bill" is selling you a fabricated number. Size it against your traffic before you believe it.

How to spot it in your code

You do not need to run anything to find this — it is legible in the source. Look for:

  • Long, essay-style tool descriptions. A tool whose description runs several sentences, restates the parameter list in prose, includes worked examples, or apologizes for edge cases is carrying tokens the model re-reads every turn.
  • Many tools registered at once. Cost scales with the number of tools in the list. A dozen tools, each with a paragraph of docs, is a large fixed block on every request. (The broader "big block on every request" dynamic is in the hidden cost of long context windows.)
  • Parameters with prose docstrings. JSON-schema description fields that read like documentation — "The user's full name, including title if provided, used for the greeting..." — multiply across every parameter of every tool.
  • Re-sent on every turn. The multiplier is the loop. Find where the tool list is built and confirm whether it is passed fresh into each agent iteration. If it is, the schema cost is paid once per turn, not once per task.

The tools that cost the most are the verbose ones multiplied by the longest agent loops. That intersection is where to look first.

How to fix it honestly

The fix is the boring, correct one: trim the documentation to what the model needs to call the tool correctly, and nothing more.

  1. Write a concise instruction per tool. A clear name, one line on what it does and when to use it, and parameter names that are self-explanatory. Drop restated parameter lists, redundant examples, and prose that repeats the schema.
  2. Keep only what changes the model's behavior. If removing a sentence does not change which tool the model picks or how it fills the arguments, that sentence was paying rent for nothing. Genuine disambiguation between similar tools stays — trimming is not deletion for its own sake.
  3. Measure before and after. Count the tool-block tokens before and after the trim, and — this is the part that keeps you honest — re-run your real tool-calling cases to confirm the model still selects the right tool and fills arguments correctly. A smaller schema that breaks tool selection is not a saving, it is a regression. If you want the fuller discipline for proving a cost change holds up, it is in how to reduce LLM token costs.

Trim, verify behavior held, then size the dollar impact against your own re-send volume. In that order.

Scan your own code

This is one of the findings erabot flags: it scans your source for tool and function definitions, surfaces the oversized ones, and estimates the token weight they carry — with assumptions marked and the re-send multiplier called out rather than hand-waved into a headline percentage. It will not promise you 97% off your bill, because that number is not real for most codebases. It will show you where the bloat is so you can decide what it is worth. Start at erabot.ai.

EB
erabot.ai

Scan any codebase. See exactly what you spend on AI. Get one-click fixes.

No spam. Only launch updates and cost optimization tips.

Product

  • Product
  • CLI
  • Pricing
  • Changelog
  • Docs

Resources

  • Example audit
  • Blog
  • Case studies
  • Eval

Company

  • Security
  • Privacy
  • Terms
  • Support

© 2026 erabot.ai. All rights reserved.