The Small Model Kept Walking: What EnergyAgentBench Actually Found
EnergyAgentBench found that model rank changed with task shape, and that a small Claude model led one repetitive long-horizon workflow by refusing to skip required tool calls.
Scope note: This essay covers model selection for tool-using energy workflows, especially procedural completion versus causal diagnosis. It does not rank language models in general.
A model can fail by being too eager to look intelligent.
Give it twelve required tool calls and it makes eight, infers the remaining values, then presents a polished answer built on work it never performed. The shortcut looks like reasoning. In an auditable workflow, it is simply a missing record.
EnergyAgentBench contains a useful example of this failure. A small Claude model beat several larger, more expensive systems on one long procedural task because it kept making every required call.
The lesson is not that small models are better. The lesson is that “best model” is an unfinished sentence. Best at what operation, under which failure conditions, at what cost?
The expensive model skipped steps
EnergyAgentBench tests agents against live or current energy-infrastructure data. Its 70 task variants include electricity-price comparisons, grid-carbon analysis, technology-cost forecasts, datacenter siting, portfolio analysis, and causal diagnosis. A run may require as few as three tool calls or as many as 48.
The overall result looks conventional at first. Claude Sonnet 4.6 led the complete benchmark with a mean score of 0.900. Claude Opus 4.7 scored 0.889, and GPT-5 scored 0.887. Sonnet also cost substantially less per run than Opus in the paper’s evaluation.
Then the ranking breaks.
In the long-horizon siting family, Claude Haiku 4.5 scored 0.986. Opus scored 0.950, Sonnet 0.933, and GPT-5 0.909. The smallest model in that comparison produced the strongest result.
The task explains why. The agent had to query the same datacenter-cost tool across eight or twelve combinations of region and hardware, retain each returned value, and assemble the final ranking. Haiku completed the sequence. Larger models sometimes inferred later values from earlier calls instead of querying every required combination. Their answers could remain plausible while their trajectories became incomplete.
The smaller model behaved less like a consultant and more like a metronome. For this job, the metronome won.
One benchmark contained different jobs
Aggregate scores tempt us to treat agent capability as one vertical scale. EnergyAgentBench makes that scale visibly unstable.
Repetitive siting rewarded procedural endurance: call the tool, preserve the value, repeat without improvising. Causal grid diagnosis required something else. The agent had to compare several sources, detect a change, and resist committing to the first convenient explanation.
Those task families separated models by different amounts. The ordinary siting family produced only a 6.6-point spread. The causal family produced a 30.7-point spread between its highest- and lowest-scoring systems. A model choice that barely matters for one workflow can dominate another.
This is the deployment signal that an overall leaderboard conceals. A single rank compresses several kinds of work into one number, then asks engineering to pretend nothing was lost.
Route by failure mode
I take a practical rule from the paper: route work according to the behavior it requires and the failure it cannot tolerate.
Send repetitive, structured evaluation to a model that completes the sequence reliably. Send cross-source causal analysis to a model that checks competing explanations before it commits. If a cheap model can execute the narrow operation perfectly, using a flagship model for prestige adds cost without adding correctness.
The reverse also matters. A small model’s procedural discipline does not establish that it can perform broad causal analysis. Haiku won one 63-run task family across three seeds; Sonnet won the complete benchmark. “Haiku beats frontier models” would be a louder headline and a worse reading.
The point is a roster, not a throne.
An agent system should know which model handles each class of work, what evidence each route must leave behind, when an incomplete trajectory invalidates a polished answer, and when uncertainty requires escalation. Model selection then becomes ordinary system design: exact, revisable, and answerable to the job.
Keep the map narrow
The benchmark does not settle energy-infrastructure analysis. It covers four U.S. electricity markets. Several task families use ground truth derived from the author’s own platform and fitted cost models rather than independent commercial estimates. The causal score partly relies on text matching, which can reward a correct label even when the explanation is weak. Live data reduces stale-question contamination but makes evaluation dates harder to reproduce.
The paper is also single-authored. Its findings deserve independent replication before anyone treats the reported ranking as an operating law.
These limits do not erase the useful result. They keep it in the correct container.
A frontier model may reason broadly and still skip a required step. A smaller model may know less and complete the procedure exactly. Once the benchmark measures work rather than prestige, neither result is paradoxical.
The small model kept walking. A well-designed system notices where—and sends it there again.