A Skill Is Not More Context: What SkillsBench Measured
The latest SkillsBench evaluation found large gains from curated procedural skills, losses from self-generated skills, and a severe penalty for exhaustive documentation.
Scope note: This essay covers curated agent skills as compact procedural decision aids. It does not assess every form of documentation, retrieval, or context engineering.
A skill should make a decision easier. If it merely adds more material to read, it is documentation with better marketing.
That distinction matters because agents can consume an enormous amount of context without acquiring the one thing they need: a tested procedure for this kind of work, including the point where that procedure should not be used.
The latest revision of SkillsBench gives the distinction useful measurements. Across 87 tasks, curated skill packages raised average pass rates by 16.6 percentage points. Comprehensive documentation barely moved them.
The headline says skills work. The failures explain what a skill actually is.
More instructions did not mean more capability
SkillsBench compares matched agent runs with and without structured packages of procedural knowledge. Each task has a deterministic verifier. The agent must discover the relevant skill, apply it inside a tool-using workflow, and produce an artifact that passes inspection.
Curated skills raised the average pass rate from 33.9% to 50.5% across the current aggregate. The improvement appeared in every domain, although its size varied. Natural science and media-production tasks gained far more than mathematics and operations research.
The likely reason is concrete. Some work depends on brittle formats, domain conventions, calibrated defaults, or exact tool sequences that broad model training does not reliably preserve. A compact procedure can place those decisions beside the task.
But the benchmark also found a limit. Tasks with one skill gained 18 points. Tasks with two or three gained 19. Tasks with four or more gained only 10.1. Comprehensive documentation gained 0.7.
An encyclopedia is not a skill. It is weather.
A useful skill includes the exit
Thirteen tasks became worse when skills were available. The researchers inspected the paired trajectories and found three recurring failures.
Some skills imposed a heavy pipeline on a task that needed a direct answer. Some displaced a stronger strategy the model already knew. Others sent the agent toward a solver or toolchain it could not debug within the available time.
The common defect was not lack of detail. It was lack of judgment. The procedure declared one correct route without naming its cost, its applicability boundary, or the conditions that should trigger a lighter fallback.
This is the part of the result I find most useful. A good skill does not merely say what to do. It says when the method applies, what evidence should exist after each consequential step, and when to stop following it.
The exit is part of the procedure.
Self-generated instructions lost to no instructions
SkillsBench also tested a tempting shortcut: ask the agent to generate a skill package from the task, then solve the task through that package.
Every tested self-generation configuration performed below its no-skill baseline. The reported losses ranged from 8.1 to 11.5 percentage points. Curated skills added between 18.2 and 24.8 points on the same configurations.
The failure had several sources. Some generated procedures were confidently wrong. Some packages were never discovered by the solving agent. The authoring phase also consumed time that should have gone into the task.
This does not prove that an agent cannot help write a skill. It proves that generation is not curation. Someone still has to compare the procedure against source material, test it on real work, remove the parts that create friction, and revise it when failure exposes a bad assumption.
The skill earns authority through use. YAML is not a sacrament.
The surrounding system still matters
A skill does not enter an empty model. It enters a harness with its own discovery rules, prompt structure, tools, filesystem conventions, and execution budget. The same model can use the same procedural knowledge differently when any of those conditions change.
That means a portable skill needs more than good prose. Its trigger must be discoverable. Its files must live where the harness looks. Its script must run in the available environment. Its validation must inspect the artifact that actually matters. A perfect instruction hidden from retrieval has the practical value of a sealed manual under a collapsed building.
The benchmark covers terminal-based container tasks, so its results should not be stretched casually across GUI agents or long multi-agent systems. Its curated skills also represent a high-quality tier, not the median package found in the wild.
Craft, then proof
A useful test follows: what decision does the skill preserve, what failure does it prevent, and what evidence would show that it helped?
If those questions have no exact answers, the package is probably context wearing a uniform.
SkillsBench provides evidence that procedural knowledge can partly separate capability from model scale. Smaller models equipped with good skills sometimes matched larger models operating without them. The gain did not come from volume. It came from compact, discoverable, bounded judgment.
A skill is not more context. It is craft made executable—and then forced to prove itself.
