erabot.ai
ProductPricingCase studiesEvalBlogDocs
Book a demo

We scanned gpt-researcher expecting waste. It's actually well-built — here's what good looks like

Second in the series. Last time (OpenAI's Swarm) we found a legacy model on a trivial task. This time we pointed erabot at a top autonomous research agent and went in expecting a mess — research agents fan out into dozens of LLM calls per run. We were mostly wrong, and that's the more useful story.


Why this one is different

A cost scanner that only ever says "you're overpaying" is just a hype machine. The honest answer is sometimes "this is fine." gpt-researcher (20k+ stars, a go-to autonomous research agent) is a case of someone getting the hard part right. Here's what erabot found, and the one thing worth a second look.

The thing most scanners miss: the models aren't in the code

gpt-researcher never writes model="gpt-5.4" at a call site. It uses a three-tier config, and every call reads self.cfg.smart_llm_model / fast_llm_model / strategic_llm_model. A regex scanner sees model=self.cfg.smart_llm_model and gives up ("model: unresolved") — and lowballs the whole repo. erabot follows the indirection to the actual defaults:

# gpt_researcher/config/variables/default.py
"FAST_LLM":      "openai:gpt-5.4-mini",   # lightweight tasks
"SMART_LLM":     "openai:gpt-5.4",        # report writing (2k+ words)
"STRATEGIC_LLM": "openai:gpt-5.4",        # planning / reasoning

That resolution is the whole game — the cost of an agent lives in which tier each call uses, and you can't see it without resolving the config.

The verdict: the tiering is right

This is the thing most agents get wrong, and gpt-researcher gets it right:

  • Fast tier (gpt-5.4-mini) → lightweight work (e.g. the image-prompt calls in skills/image_generator.py).
  • Smart tier (gpt-5.4) → the actual report generation (actions/report_generation.py) — 2k+ word synthesis that genuinely needs the big model.
  • Strategic tier (gpt-5.4, with a tunable REASONING_EFFORT) → query planning and deep-research orchestration.

Using an expensive model for long-form report writing isn't waste — it's the job. The split is deliberate and correct. Credit where it's due.

The one candidate

gpt_researcher/skills/curator.py ranks and filters scraped sources by relevance/credibility and returns a structured JSON list — and it runs on the smart tier:

# curator.py — ranks sources, returns JSON
response = await create_chat_completion(
    model=self.researcher.cfg.smart_llm_model,   # <-- SMART (gpt-5.4) for a ranking task
    ...
    max_tokens=8000,
)

Ranking/filtering a list and emitting JSON is a textbook narrow task — the kind the fast tier (gpt-5.4-mini) usually handles fine. Routing this one call to fast_llm_model is a candidate optimization (verify it holds ranking quality on your data before shipping — that's the whole point of candidates).

The honest caveat that kills the clickbait: curation is off by default (CURATE_SOURCES: False). It only costs you if you turn it on. So this isn't "gpt-researcher overpays" — it's "if you enable curation, consider the fast tier for it." We're not going to inflate that into a headline it doesn't deserve.

On dollars: gpt-5.4 is new enough that we won't pin an exact figure we can't stand behind — but across the GPT-5 family the smart→mini gap is roughly an order of magnitude, so on a curation-heavy run it's a real saving for the users it applies to.

The actual lesson

Tiering is the thing. An agent's bill is decided by which model does each job — the cheap tier for narrow work (rank, extract, route, classify), the expensive tier for the hard generation. gpt-researcher's architecture makes that choice explicit and mostly gets it right. If yours doesn't have a fast/smart split at all — every call hitting one frontier model — that's where the real money is, and it's invisible until you resolve the config.

Scan your own agent (free, and it resolves your config)

  • Claude Code: claude plugin marketplace add erabot-ai/erabot-plugin → /erabot:scan. It runs on your own agent, follows your config and SDK wrappers to the real models, and reads the loop — where the money hides.
  • Want the swaps verified (proof each holds quality on your real traffic, plus apply-ready diffs)? → erabot.ai.

How erabot stays honest: we resolve config-set models instead of giving up, we mark every assumed number, a swap is a candidate until shadow-tested, and when a codebase is well-built we say so. We'd rather be trusted than impressive.

EB
erabot.ai

Scan any codebase. See exactly what you spend on AI. Get one-click fixes.

No spam. Only launch updates and cost optimization tips.

Product

  • Product
  • CLI
  • Pricing
  • Changelog
  • Docs

Resources

  • Example audit
  • Blog
  • Case studies
  • Eval

Company

  • Security
  • Privacy
  • Terms
  • Support

© 2026 erabot.ai. All rights reserved.