What the Prompt-Caching Study Found: Put the Stable Things First
A cross-provider study found large prompt-caching savings, uneven latency gains, and one durable rule for agent architecture: preserve the stable prefix.
Scope note: This essay covers prompt caching for repeated stable prefixes in long-running agents. Provider prices, cache rules, and latency behavior remain implementation-specific and time-bound.
A timestamp can be expensive without costing a token of its own.
Place that changing value at the beginning of an agent’s system prompt and every stable token after it may become new work. Place it after the reusable instructions and the provider may reuse thousands of tokens of prior computation.
The information has not changed. Its order has.
Don’t Break the Cache measures the cost of that order across OpenAI, Anthropic, and Google models. The reported savings reach 80%. One naive caching strategy also made a tested model slower.
The durable lesson is not “turn caching on.” It is that a cache has a boundary, and agent architecture decides whether that boundary survives.
Long-running agents repeat stable material
A tool-using agent sends the same system prompt, tool definitions, policies, and operating rules on turn after turn. The conversation changes. Search results accumulate. Files appear. Tool outputs arrive. The stable prefix returns anyway.
Prompt caching lets a provider reuse computation for an exact repeated prefix. That final word matters. A cache does not reward conceptual similarity. It rewards the same tokens in the same order.
The researchers tested more than 500 DeepResearchBench sessions built around a 10,000-token system prompt and repeated web-search calls. They compared no caching, full-context caching, system-prompt-only caching, and a strategy that excluded dynamic tool results.
Across the tested provider implementations, the cost reductions were substantial. GPT-5.2’s exclude-tool-results strategy cut cost by 79.6%. System-prompt caching cut cost by 78.5% for Claude Sonnet 4.5, 45.9% for GPT-4o, and 41.4% for Gemini 2.5 Pro.
Those percentages belong to the models, prices, prompts, and cache rules tested in January 2026. They will age. The boundary will remain.
Saving money can still add delay
Full-context caching reduced GPT-4o’s cost by 47.8% in the study. It also made time to first token 8.8% worse than the uncached baseline. Caching only the system prompt improved that latency by 30.9%.
The reason is structural. Stable instructions can produce repeated cache hits. Dynamic tool calls and results change from turn to turn. Writing them into the cache can create overhead without creating future reuse. One strategy reduces billed input while making the reader wait longer for the first token.
The pattern differed by provider. Claude Sonnet 4.5 improved latency under every tested cache mode. Gemini showed a small latency regression under the exclude-tool-results strategy. GPT-5.2’s best cost result and best latency result came from different configurations.
There is no universal cache switch. There is a workload, a prefix, and a measurement.
Put volatile state after the gate
A changing timestamp, session identifier, user-specific value, reordered tool definition, or dynamically discovered schema can invalidate the stable material placed after it.
The paper’s practical rule is wonderfully literal: keep stable material first. Put session state and other volatile values after the reusable prefix. Treat tool results as dynamic unless the system has a concrete reason to expect them again.
Prompt length matters too. The researchers tested prompts from 500 to 50,000 tokens. Provider caching minimums ranged from 1,024 to 4,096 tokens. Below those thresholds, the feature could not activate. At 50,000 tokens, the tested cost savings reached 54% to 89%, depending on the model.
The length of the reusable prefix mattered more than the number of tool calls.
That finding reaches beyond billing. Dynamic tool discovery may improve flexibility while destroying deterministic schema order. Aggressive history rewriting may save context while invalidating a reusable prefix. A single convenience in one layer can erase efficiency in another.
Context engineering and cost engineering are the same operation viewed from different invoices.
Measure price and latency separately
The paper finds much more variance in latency than in cost. Token categories and published rates make cost comparatively stable. Time to first token crosses network conditions, server load, account tier, geographic routing, and infrastructure no customer can inspect.
I would use the reported cost direction as a design hypothesis, not import the latency percentages as production truth. The real workload needs its own measurement: cache reads, cache writes, billed tokens, time to first token, and complete response time.
The paper also notes a security concern. Shared caches can create timing side channels that reveal whether content was previously cached. Efficiency does not dissolve isolation requirements. It merely arrives with another invoice.
Information order now carries cost
A long system prompt is not automatically waste. If its instructions remain stable and useful, that prompt may become the cheapest repeated part of the agent. A tiny volatile value placed before it can make the whole object expensive again.
That is the result I keep: the order of information has acquired financial weight.
Separate stable instructions from session state. Keep tool schemas deterministic where the task permits it. Place volatile material after the reusable prefix. Then test the system that exists, not the benchmark that resembles it from a polite distance.
Put the stable things first. The cache is literal-minded. A rare virtue.