Enterprise Search Needs to Know When to Look Again
AgenticRAG lets a reasoning model search, open, inspect, and summarize documents repeatedly, improving difficult retrieval at a real cost in tokens and time.
Scope note: This essay covers an agentic layer tested on three public benchmarks and informed by pre-production enterprise work. It does not report a full production deployment or prove that iterative retrieval is best for every query.
Most enterprise search systems get one chance to choose the evidence. A weak first search becomes a polished final answer.
Microsoft researchers built AgenticRAG to change that sequence. Instead of giving a model one fixed bundle of search results, the system lets a reasoning model search again, find a term inside a document, open more of that document, and summarize what it has learned before answering.
The idea is simple: let the model behave more like a researcher and less like a passenger handed a folder through a slot.
Search becomes a loop
Traditional retrieval-augmented generation usually runs retrieval first and generation second. If the search stack misses the useful document, the model cannot recover. If the returned passage is too narrow, the model may never see the section that changes the answer.
AgenticRAG adds four tools on top of an existing enterprise search system: search, find, open, and summarize. The model decides when to use them. It can cast a broad search, inspect a promising document, move to a specific section, and return to search when the evidence is thin.
This matters for questions that cross several documents or use language that does not match the source material neatly. It also preserves the existing search index and access controls. The approach does not require a new embedding model, a custom knowledge graph, or training on private company documents.
The gains were large on difficult benchmarks
On BRIGHT, a benchmark built around reasoning-heavy retrieval, the system reached 49.6 percent recall at the first result. That was 21.8 percentage points above the best embedding baseline reported in the comparison.
On WixQA, which contains procedural support questions, factuality reached 0.96. On FinanceBench, the system answered 92 percent of questions correctly, close to the 94 percent achieved when the model received the correct evidence directly.
The authors’ ablation work found that the largest improvement came from moving away from one-shot search. Tools for opening documents and searching within them added depth. Issuing several search variants at once reduced the number of tool calls needed to reach similar recall.
The result is not that search no longer matters. The agent still depends on the existing search system to surface candidates. The change is that a first imperfect result is no longer final.
Better research costs more
On BRIGHT, the agentic system used 2.6 times as many tokens as a one-shot search. On FinanceBench, the ratio was 7.8. The system performed far better, but the extra search, reading, and reasoning were not free.
The hardest benchmark slice required finding many related documents rather than a few decisive ones. AgenticRAG remained weak there. Its coarse-to-fine behavior favored a small set of high-value sources.
That points toward routing, not replacement. A straightforward question should use fast retrieval. A complex, multi-part question may justify the deeper loop. The system should decide which path to use before it spends a small winter reading the archive.
Production still has to prove itself
The evaluation uses public benchmarks. The design was informed by pre-production deployments, but the paper does not report user outcomes, latency under broad production load, access-control failures, or maintenance costs in a live organization.
Benchmark correctness also does not settle whether employees can inspect the sources, understand uncertainty, or catch a wrong synthesis. The final interface still needs visible citations and a clean route back to the document.
Even with those limits, AgenticRAG makes an important correction. Grounding is not a single retrieval event. For difficult questions, it is a process of looking, checking, changing direction, and knowing when the evidence is enough.
Enterprise search does not need to pretend the first answer is fate.