Cheap but confused?
0. The four-minute film
Narration note: the film is phrase-anchored to a timing database produced by video/film_tts.py. The ElevenLabs key provided for this project was rejected (HTTP 401) on every call, so this build carries a placeholder macOS voice at the same loudness target; re-running the script with a valid key regenerates the Matilda narration and re-times every scene.
1. Every agent now obeys one rule: never touch the start of the prompt
Providers bill input tokens they have already processed at about a tenth of the list price. The discount applies only when a request begins with exactly the bytes of an earlier request. The guidance that followed is specific: put the system prompt and the tool declarations first and never change them, append every new observation at the end, mask tools instead of removing them, never reorder or delete earlier blocks. We call this the cache-optimal layout.
The layout has a side effect that nobody has priced. Nothing is ever removed, so when a value changes, the old value stays in the context next to the new one. A model that answers from its context must then prefer the later of two values for the same field. The question of this study is whether that costs accuracy with the information content held fixed, and what the cost is worth against the cache it protects.
2. A testbed where every stale value is known by construction
To blame an error on a stale value, you must know, for every question the agent answers, whether a superseded value for the queried field is in its context. Public agent benchmarks do not record this. So we built StaleTrap: three back-office domains (warehouse inventory, a shared calendar, a support-ticket queue) exposed as 29 JSON-schema tools, plus 10 messaging and HR tools that are present in every call and never needed. A local executor holds the state and returns realistic JSON, and every result declares which (record, field) values it states.
An episode is a scripted session of 6, 12, 18 or 30 tool calls with 2 to 8 question steps served with tool calling disabled. In a trap step a superseded value is sitting in the context when the question arrives. In a neutral step nothing competes. Traps come in two kinds. In an own update, the agent changed the value itself and the tool reply stated the new value. In an external update, someone else changed the record and the agent only ever saw the new value as one row inside a listing it was asked to fetch. The grid is 144 episodes, 720 questions (360 trap, 360 neutral), 2,376 tool calls, frozen with a SHA-256 manifest before the first scored call.
3. Four layouts that carry the same information
Each turn, the runner rebuilds the message list from the transcript so far under one rule. L1 is the cache-optimal layout: arrival order, newest observation last, tools masked rather than removed. L2 keeps the same blocks but reorders the earlier ones by how many record identifiers they share with the current turn, most relevant last, which rewrites the prefix every turn. L3 keeps arrival order and redacts, in place, every value that a later result restated; it is an oracle pruner and an upper bound for any real one. L4 is L1 plus a pointer-only note kept on every user turn: the goal, the step, and for every record with more than one observation which step is current. The note never contains a value.
4. Own updates never fail; external updates lose a third of the answers
On Gemini 3.1 Flash-Lite, which ran the full grid, every layout scores 100.0% on own-update traps and 100.0% on neutral steps. The whole effect lives in external updates. There the append-only layout answers 65.5% correctly, the reordered layout 70.5%, the pruned layout 96.0% and the note 73.5% (200 external-trap questions per layout).
Over all 360 trap questions the paired contrasts are: L1 minus L3 -0.177 [-0.230, -0.128], p < 0.0001; L1 minus L2 -0.036 [-0.079, 0.006], p = 0.116; L4 minus L1 +0.073 [0.031, 0.120], p = 0.001. Neutral steps differ by exactly zero between every pair of layouts. The gap does not grow with episode length (slope +0.0006 per call, p = 0.847): it is already 22.2% at six calls.
5. Reading the wrong answers: it is the stale value, read back
We hand-coded 40 wrong trap answers from the main grid, each with the full observation history of the queried field. 34 are a superseded value read back: typically the agent's own earlier write, with the later listing row that overtook it ignored. 3 are threshold questions where the fresh quantity equals the threshold, so a stale read and a strict-inequality misreading give the same wrong answer. 2 are questions where the agent had skipped the scripted listing call, so the fresh value never entered its context (both under L3). 1 is a derived number that matches no observation. Restricting every accuracy to questions whose scripted thread steps were all executed moves no layout by more than a point.
6. The bill: cheapest per episode is not cheapest per correct episode
Per episode, billed by the provider, the append-only layout is the cheapest at 1.05c; the pruned layout costs 1.16c (p < 0.0001), the reordered layout 1.35c and the note 1.60c. Per correct episode the order flips: pruned 1.22c, append-only 1.52c, reordered 1.85c, note 2.07c.
We then repriced every cached token at multipliers of the input price from 0 (reads free) to 1 (no discount). The cheapest layout per correct episode at each multiplier is:
| 0.0x: L3 | 0.01x: L3 | 0.025x: L3 | 0.05x: L3 | 0.1x: L3 | 0.25x: L3 | 0.5x: L3 | 0.75x: L3 | 1.0x: L3 |
At every price the pruned layout wins. The pre-registered prediction was that a threshold exists below which the cache-optimal layout becomes cheapest per correct answer. On this provider it does not, for a reason the next section shows.
7. Where the cache reads actually come from
Google's cache is implicit and best-effort. Across the main grid the cached share of prompt tokens is 0.71 under L1 and 0.68 under L3, a difference of 0.032 [0.018, 0.045]. Most of that share is the 5,100-token static prefix. Counting only the history (everything after the prefix), the cached share is 0.24 under L1 and 0.15 under L3. The layout that caching rewards is barely rewarded for its history on this provider, so breaking the prefix costs little. Two more facts shape the bill. No-tool question turns are served from cache far less often than tool turns (hit rate 0.51 against 0.93 under L1), because disabling tool choice changes the request. And the L4 note, 150 tokens per turn that must stay in the history to keep the prefix stable, raises prompt tokens per episode by 69 percent over L1.
The plan assigned a cache on/off control to Anthropic, whose documentation promises output-neutral caching; that provider was unreachable on this workspace's key. The substitute uses the best-effort cache itself: the same question, under the same layout, served from cache in some runs of the repeat battery and not in others. Paired within question, the difference in trap accuracy is +0.000 over 11 questions that saw both outcomes (p = 1.000).
8. Three Gemini generations
Gemini 2.5 Flash (older, thinking on, 72 tasks) and Gemini 3.6 Flash (newest, 36 tasks) bracket the main model. External-trap accuracy under the append-only layout rises with generation, 40% to 66% to 80%, and under pruning it is 84%, 96% and 100%. The gap shrinks (27, 18, 11 points; label permutation p = 0.035 over all tasks, p = 0.092 on the 36 tasks common to all three) and does not close. The pointer note does more for the older model: on 2.5 Flash it recovers 79% of the gap and lifts external-trap accuracy from 40% to 75%. On every model pruning is the cheapest layout per correct episode, at the billed price and at every counterfactual cache-read price; on 3.6 Flash the implicit cache never serves the history at all, so its two layouts cost the same per episode.
| Model | tasks | external traps, L1 | L3 | L4 | own traps, L1 | cents per correct, L1 | L3 |
|---|---|---|---|---|---|---|---|
| Gemini 2.5 Flash | 72 | 40.0% | 84.0% | 75.0% | 98.8% | 2.71c | 1.78c |
| Gemini 3.1 Flash-Lite | 144 | 65.5% | 96.0% | 73.5% | 100.0% | 1.52c | 1.22c |
| Gemini 3.6 Flash | 36 | 80.0% | 100.0% | -- | 100.0% | 4.21c | 3.67c |
9. Five pre-registered predictions, scored
| Direction (logged 2026-10-10, before any call) | Verdict | Evidence |
|---|---|---|
| D1. L1 below L3 on stale-trap steps; gap grows with length; no gap on neutral steps | confirmed in two parts of three | L1 minus L3 = -0.177, p < 0.0001; neutral gap exactly 0; length slope +0.0006 per call, p = 0.85 (not supported) |
| D2. L2 does not beat L1; its cached share falls below 0.20 and its cost rises 3x; L2 dominated on the frontier | half confirmed | L1 minus L2 = -0.036, p = 0.11; L2 dominated; cached share 0.62 (not below 0.20); cost +28% (not 3x) |
| D3. L4 recovers at least half of the gap, keeps 80% of L1's cached share, sits on or inside the frontier | not supported | recovery 0.42 [0.21, 0.60] over all traps, 0.26 on external traps; cached share ratio 1.00; L4 dominated (prompt tokens +69%) |
| D4. The L1 vs L3 gap shrinks with model capability | confirmed in direction, weak interaction evidence | L1 minus L3 gap by generation: -0.27 (2.5 Flash), -0.18 (3.1 Flash-Lite), -0.11 (3.6 Flash); label permutation p = 0.035 over all tasks, 0.092 on the common 36; the cheapest model by price sits in the middle, so the 'cheapest model' clause is not supported |
| D5. L3 cheapest per correct task when cache reads are expensive; L1 or L4 cheapest below a read-price threshold | reversed | L3 cheapest per correct episode at every read multiplier from 0 to 1; no threshold |
10. What was known, what is new
Known: models use the start and end of a long context better than the middle; the latest of several revisions of a fact is retrieved less reliably as revisions accumulate; agents answer from observations the world has overtaken; pruning or masking old observations changes accuracy in a model-dependent way; prompt caching cuts the cost of long sessions by 40 to 80 percent and the discount is lost when earlier bytes change.
New: a content-fixed manipulation of layout (order only, content only, recitation only) in a multi-turn tool-using agent, scored against provider-reported cached tokens and billed dollars, with every question annotated as a stale-trap step (own or external) or a neutral step, and a per-model answer to whether a cache-read price exists at which the cheapest layout per call stops being the cheapest per correct answer. The nearest neighbours vary content (compression) rather than layout, report cost without accuracy, or price tokens at list rates without a cache term. The paper's Section 2 lists them.
11. What this does not show
- One provider family. The plan froze three families; the workspace key allowed only Google. Every number is a Google number, and the cost conclusion depends on Google's best-effort cache rarely serving the history. A provider with explicit breakpoints and a 90 percent intra-turn hit rate would give the append-only layout a larger cost advantage, and a price threshold could then exist.
- L3 is an oracle. It redacts exactly what a later result restates. No deployed pruner has that oracle; L3 is the size of the prize, L4 is what a pointer-only note collects without one.
- A synthetic testbed. Scripted episodes, small domains, exact-match questions. The external arm on BFCL v3 multi-turn found no layout effect on 50 tasks; its failures are capability failures, not stale reads, so the mechanism has not been shown on a public benchmark.
- Noise. A trap outcome flips between identical temperature-0 runs in 11 percent of cells; the paired tests over 144 tasks absorb this, single cells do not.
- The audit is not blind. The agent that ran the study coded the 40 wrong answers with the layout visible; the codes are mechanical and ship with the release.
12. Reproduce it
Everything is in the repository: the frozen testbed with its manifest, the executor, the four layout serialisers, the runner, every raw episode log with per-call provider usage and billed cost, the single analysis script, the audit, the figure and film builders, and the spend log. Metered spend for the whole study, including pilots, calibration and discarded runs, was $18.42.