We scanned gpt-researcher expecting waste. It's actually well-built — here's what good looks like
Second in the series. Last time (OpenAI's Swarm) we found a legacy model on a trivial task. This time we pointed erabot at a top autonomous research agent and went in expecting a mess — research agents fan out into dozens of LLM calls per run. We were mostly wrong, and that's the more useful story.
Why this one is different
A cost scanner that only ever says "you're overpaying" is just a hype machine. The honest answer is sometimes "this is fine." gpt-researcher (20k+ stars, a go-to autonomous research agent) is a case of someone getting the hard part right. Here's what erabot found, and the one thing worth a second look.
The thing most scanners miss: the models aren't in the code
gpt-researcher never writes model="gpt-5.4" at a call site. It uses a three-tier config, and every call reads self.cfg.smart_llm_model / fast_llm_model / strategic_llm_model. A regex scanner sees model=self.cfg.smart_llm_model and gives up ("model: unresolved") — and lowballs the whole repo. erabot follows the indirection to the actual defaults:
# gpt_researcher/config/variables/default.py
"FAST_LLM": "openai:gpt-5.4-mini", # lightweight tasks
"SMART_LLM": "openai:gpt-5.4", # report writing (2k+ words)
"STRATEGIC_LLM": "openai:gpt-5.4", # planning / reasoning
That resolution is the whole game — the cost of an agent lives in which tier each call uses, and you can't see it without resolving the config.
The verdict: the tiering is right
This is the thing most agents get wrong, and gpt-researcher gets it right:
- Fast tier (
gpt-5.4-mini) → lightweight work (e.g. the image-prompt calls inskills/image_generator.py). - Smart tier (
gpt-5.4) → the actual report generation (actions/report_generation.py) — 2k+ word synthesis that genuinely needs the big model. - Strategic tier (
gpt-5.4, with a tunableREASONING_EFFORT) → query planning and deep-research orchestration.
Using an expensive model for long-form report writing isn't waste — it's the job. The split is deliberate and correct. Credit where it's due.
The one candidate
gpt_researcher/skills/curator.py ranks and filters scraped sources by relevance/credibility and returns a structured JSON list — and it runs on the smart tier:
# curator.py — ranks sources, returns JSON
response = await create_chat_completion(
model=self.researcher.cfg.smart_llm_model, # <-- SMART (gpt-5.4) for a ranking task
...
max_tokens=8000,
)
Ranking/filtering a list and emitting JSON is a textbook narrow task — the kind the fast tier (gpt-5.4-mini) usually handles fine. Routing this one call to fast_llm_model is a candidate optimization (verify it holds ranking quality on your data before shipping — that's the whole point of candidates).
The honest caveat that kills the clickbait: curation is off by default (CURATE_SOURCES: False). It only costs you if you turn it on. So this isn't "gpt-researcher overpays" — it's "if you enable curation, consider the fast tier for it." We're not going to inflate that into a headline it doesn't deserve.
On dollars: gpt-5.4 is new enough that we won't pin an exact figure we can't stand behind — but across the GPT-5 family the smart→mini gap is roughly an order of magnitude, so on a curation-heavy run it's a real saving for the users it applies to.
The actual lesson
Tiering is the thing. An agent's bill is decided by which model does each job — the cheap tier for narrow work (rank, extract, route, classify), the expensive tier for the hard generation. gpt-researcher's architecture makes that choice explicit and mostly gets it right. If yours doesn't have a fast/smart split at all — every call hitting one frontier model — that's where the real money is, and it's invisible until you resolve the config.
Scan your own agent (free, and it resolves your config)
- Claude Code:
claude plugin marketplace add erabot-ai/erabot-plugin→/erabot:scan. It runs on your own agent, follows your config and SDK wrappers to the real models, and reads the loop — where the money hides. - Want the swaps verified (proof each holds quality on your real traffic, plus apply-ready diffs)? → erabot.ai.
How erabot stays honest: we resolve config-set models instead of giving up, we mark every assumed number, a swap is a candidate until shadow-tested, and when a codebase is well-built we say so. We'd rather be trusted than impressive.