The State of the Harness

A few days ago Drew Breunig published The State of Context Engineering, the write-up of his AGNTCon talk, and I found myself wanting to check every part of it against the papers. That turned into a proper state of the art on agent harnesses, and then into a deck of 62 slides called “The State of the Harness”.

A note on how this was made. I put this state of the art together for my own use, to understand where agent harnesses are heading; I’m sharing it here because it might be useful to others too. Most of this work was automated. The literature review was generated with Claude’s Research feature (Anthropic), the slide illustrations were generated by Claude too, and Claude also checked the sources and their links. I then read everything myself and, with my modest level in LLMs and agent harnesses, the claims look plausible to me. That is not the same as being right: some numbers, attributions or readings of a paper may well be wrong. If you spot an error, I’d be grateful if you told me, and I’ll correct it.

Below are the slides and a lightly edited version of what I say over each one. It’s long, so feel free to skip around: every slide has its own anchor.

Opening

Dark cover slide titled The State of the Harness. On the right, three nested squares: the model in orange at the centre, inside an opaque provider layer in dark grey, inside the harness in blue, labelled loop, context, tools, memory, guardrails.

Welcome. This is my attempt at a state of the art on agent harnesses: what the 2026 research literature actually says, and where the problem is likely to move in 2027.

The picture on the right is the whole talk in one image. The model sits at the centre and our harness code wraps around it. In 2026 a third layer slid in between the two: the provider’s API, which now does part of the context management for us, and does it in a way we can’t see into. Keep that middle layer in mind, because most of what follows comes back to it.

The model as an orange disc at the centre, connected to six blue cards: the loop, context assembly, tools and skills, state and memory, guardrails, evaluation.

First, a bit of vocabulary. An agent is a model inside a harness. The model only predicts the next token; everything else is the harness. There’s the loop that keeps calling the model (while not done: think, act, observe). There’s the code that assembles what goes into the context window, and in what order. There are the tools and skills the model can use, the state and memory that persist between turns and sessions, and the guardrails around budget, stopping, error handling and permissions. Finally there’s evaluation, the part that tells you whether any of it is working.

Breunig defines harness engineering as designing the scaffold, orchestration and tooling that elicits desired outputs from an agentic application. I like that definition, and I’ll use it throughout.

Sources: Breunig, The State of Context Engineering.

Four cards summarising Breunig's map: six failure modes, six tactics, one rule with an arrow from probabilistic to deterministic, and one prediction with a crystal ball.

My starting point is Breunig’s talk, given at AGNTCon and published on his blog on 24 October. It’s a beautifully simple map in four parts.

There are six failure modes: the four context failures from 2025 (poisoning, distraction, confusion and clash) plus two new ones that show up with long-horizon agents, fighting the weights and context inertia. There are six tactics to fix them: augmentation, tool loadout, quarantine, pruning, summarisation (or compaction) and offloading. There’s one rule: our job is to choose where to be deterministic and where to be probabilistic, and the trend moves context assembly from model calls into code. And there’s one prediction for 2027: over time, memories become conflicted and messy, and strategic forgetting is unsolved. I took each of those four pieces and checked it against the 2026 literature.

Sources: Breunig, The State of Context Engineering.

A row of six large numbers, 114, 99, 22, 67, 16 and 9, above the four research tracks and the legend of evidence badges.

Here’s how I went about it. I gathered 114 sources, 99 of them from 2026, and organised the work into four research tracks: forgetting and memory, context structure, harness versus model, and teams and evaluation. On top of that I wrote down seven hypotheses, H1 to H7, and tested each one; the verdicts come at the end.

Here are the seven, so you can keep them in mind along the way:

H1. The append-only transcript stops being the source of truth.
H2. Forgetting becomes a harness primitive with an explicit policy.
H3. Exact memory belongs to deterministic storage; the working context stays partial and sorted.
H4. The organisation of information matters as much as its length.
H5. Part of context management migrates into the weights, creating new couplings.
H6. The reusable unit moves from the skill to the agent team.
H7. The bottleneck becomes discovering what already exists.

The corpus breaks down into 22 peer-reviewed works (workshops included), 67 preprints, 16 practitioner sources such as blogs, talks and repos, and 9 lab reports, lab data, official docs and vendor repositories. One rule held throughout: every number carries its level of evidence, and you’ll see the badges at the bottom of every slide. A warning up front, too. This field moves so fast that seven of the central works were between three days and six weeks old when I wrote this, and none of those seven had been peer-reviewed.

Before and after table: Breunig's numbers in grey (6 failure modes, 6 tactics, probabilistic to deterministic, memories will get messy) against the 2026 literature in blue (12, 18, inspectable to opaque, already measured).

So what did the 2026 research do to that map? Here’s the scorecard.

Failure modes go from six to twelve, and the six new ones aren’t variants of the old ones: they have their own symptoms and their own fixes. The six tactics remain the right categories, but the toolbox now holds eighteen techniques, and pruning, compaction and loadout have moved inside the providers’ APIs. The rule, push work from probabilistic to deterministic, is true but incomplete; the bigger shift is from inspectable to opaque. And the prediction about messy memories had already been named and measured in May 2026, with two narrow governance recipes that work.

Dark punchline slide: the frontier isn't moving from probabilistic to deterministic, it's moving from inspectable to opaque. Below, an Inspectable card with an eye, an arrow, and an Opaque card with a padlock.

If you remember one sentence from this post, I’d like it to be this one.

Breunig’s framing says our job is to move work from probabilistic model calls into deterministic code. That’s still happening. The larger movement in 2026, though, runs along a different axis. Context management is migrating into the model’s weights, through learned compression and KV states, and into the provider’s API, through signed and encrypted compaction blocks. Those are places where neither you nor the model can read what’s going on. What we’re losing there is observability, and I hold that thesis with high confidence.

Coming from explainability work, I read this less as a reason to give up on inspection than as a reason to invest in it: provider-side opacity is a governance choice, and weight-side opacity is what interpretability research exists for.

Sources: Breunig, The State of Context Engineering.

Ten chapter cards in a five by two grid, each with a number, an emoji, a name and a question.

Here’s the route: ten stops. We’ll start with a quick recap of how we got here, then the failure modes and the toolbox. After that come the deep dives: forgetting, the shape of the context, who should manage it (harness, model or provider), long-lived memory, teams of agents, and whether we can measure any of this at all. We’ll finish with what a 2027 harness should look like and what to do right now.

01 How we got here

Horizontal timeline from 2023 through late 2024, 2025 and 2026 to 2027, one coloured dot per era, each with a What moved box underneath.

A quick recap. In 2023 we did prompt engineering: information was chosen by hand and written into a single prompt. Nothing had moved yet, because context was written rather than assembled.

In late 2024 came the long-context era. Windows went from 128K to a million tokens, with ten million announced, and “RAG is dead” was the slogan of the moment. Selection became optional, and four failure modes appeared. In 2025 we called it context engineering: the system assembles the context dynamically, the six tactics settled, and assembly left the written prompt for harness code.

In 2026 it became harness engineering. Models are now trained to drive the harness themselves, and METR’s 50% time horizon went from 3.4 hours for GPT-5 to roughly 12 hours for Opus 4.6 (recomputed from METR’s own data file). Steering moved to the model, which brought fighting the weights and context inertia, and durable state started to leave the conversation. As for 2027, the context primitives are moving to the provider, signed and opaque, and the questions become strategic forgetting, model and harness portability, and whether our measurements are even valid. Every gain fixes the previous problem and creates a new one a level higher. The vocabulary changes; the job of deciding what belongs in code and what belongs in the model stays.

Sources: METR, Time Horizon 1.1 and its data files.

Three columns in large type: 2024, just put everything in the context; 2026, just teach the model to drive the harness; 2027, just question marks. Each column carries the emoji of the failure modes it brought.

Each era came with a shortcut. In 2024 the pitch was to put everything in the context and let the model sort it out. That brought poisoning, distraction, confusion and clash. In 2026 the pitch became to teach the model to drive the harness, and that brought fighting the weights and context inertia.

So what’s the shortcut of 2027? I’ll leave that hanging for now. We’ll come back to it in chapter six.

Sources: Breunig, The State of Context Engineering.

02 Failure modes

A four by three grid of failure-mode tiles: Breunig's six in grey, and six new ones on a yellow tint marked NEW, with emoji for a hole, a sleeping face, a repeat button, a wilted flower, a melting face and a zombie.

Breunig’s map had six failure modes. The 2026 corpus documents twelve, and the six new ones come with their own symptoms and their own fixes.

Idea basins: independent runs stay trapped in one neighbourhood of the solution space. Premature abandonment: the agent gives up and declares a plateau long before its budget is spent. Algorithmic mode collapse: the diffs keep looking different on the surface while the mechanism underneath stops changing (that one comes from Beneath the Diff, EMNLP 2026). Library drift: an unmaintained skill library grows until injecting a skill is worse than injecting none (FAGEN@ICML 2026). Context collapse: a single end-to-end LLM rewrite of the context wipes out the accumulated playbook, from 18,282 tokens to 122 in one step, although that’s one observed transition, so n = 1. And what I think of as zombie memories, “revoked but authoritative”: a fact you’ve invalidated is still retrieved and outranks its replacement, and none of the five memory systems tested enforces revocation.

The worst part is that three of these are invisible to the usual metrics. Requirement coverage can stay above 92% while seven runs out of ten produce the wrong artefact.

Sources: Breunig, The State of Context Engineering, Beneath the Diff: Algorithmic Mode Collapse.

Left, two skill files side by side, Skill A with three green ticks and Skill B with three red crosses, crossed swords between them. Right, range bars of 93 to 99 percent alone against 61 to 62 percent with a competing skill, and a green card for the fix that works.

Let’s put a number on clash. The SkillClash paper (One Skill Takes Over, arXiv 2610.25064) loads two skills that do the same job and checks every rule with code.

Loaded alone, a skill’s rules are followed 93 to 99% of the time. Add a competing skill and that falls to 61 to 62%. It isn’t a gentle average either: one skill takes over, all of its rules are followed, and all of the other’s are dropped. Telling the model which skill has priority doesn’t fix it. What does fix it is surprisingly simple. End the skill file with one correct worked example that satisfies both skills, and takeover falls to between 1 and 15% across six models. A merged file follows 96 to 99% of the rules it can reach, and most of that gain comes from its single worked example. A wrong example doesn’t help. Worth saying: this preprint was eight days old when it was read for this review.

Sources: One Skill Takes Over.

Left, a dark code-editor card where MUST, NEVER, CRITICAL and parallel are sized by how often they appear. Right, two per-model prompt files for GPT-5.1 and GPT-5.2 with 0.929 similarity, and two code chips, model_switch and ModelDownshift.

Fighting the weights is when the model doesn’t do what you ask because you’re pushing against its training. Breunig counted the repetitions in Claude Code’s prompt, but that prompt isn’t published, so his counts can’t be checked. Instead I measured the same signature on prompts vendors do publish.

Google’s gemini-cli prompt file runs to 70,483 characters. It says MUST 34 times, NEVER 12 times and CRITICAL 6 times, and “parallel” appears 11 times in at least seven distinct places. It’s also getting louder: MUST went from 21 in the legacy variant to 34 in the current one.

OpenAI’s Codex goes further and ships one system prompt per model. Between GPT-5.1 and GPT-5.2 the line similarity is 0.929; the newer prompt adds a parallel tool-call instruction (“parallel” goes from 1 to 4) and drops 36 lines of communication guidance. Switching models mid-session needs a <model_switch> re-brief, and a compaction can be retried with the current model (CompactionReason::ModelDownshift), which tells you that a compacted state from one model isn’t guaranteed to work with another. That’s the most direct proof I know that harness and model are coupled. These counts were taken on 26 October 2026, and the repos change daily.

Sources: Breunig, The State of Context Engineering.

Three panels: two before and after bar pairs, edits falling from about 150 to about 30 lines and reasoning from about 28K to about 7K tokens, and a scatter of four coloured clusters labelled idea basins.

Context inertia used to be an intuition. It now has a signature you can measure.

First, edits shrink. In Berkeley’s experiments on AHC014, a team’s edits fall from about 150 changed lines per proposal to about 30 before its first reset. Second, and this one surprised me, reasoning shrinks too: from about 28,000 reasoning tokens per turn to about 7,000. A model trained to think long thinks less as its session grows. Third, Microsoft Research mapped the artefacts of independent runs by functional similarity and found they stay in their own localised regions, the idea basins. The 2D map agrees with 85% of held-out triplet judgements, against 50% by chance, and the localisation persists despite periodic compaction. Sources here are two preprints: Scaling Discovery Requires Forgetting (arXiv 2610.20563) and Fork-and-Flush (arXiv 2610.07447).

Sources: Scaling Discovery Requires Forgetting, Fork-and-Flush: Escaping Idea Basins in Autoresearch Agents.

A robot speech bubble quoting the agent, a cycles axis where lower is better with markers at 1,710, 1,644 and 1,077, and a ten by ten waffle chart where 70 grey squares are turns that produced nothing.

This is my favourite example of the year, a sort of 2026 equivalent of Breunig’s Pokémon tea story.

On Anthropic’s VLIW kernel task, an agent sitting at 1,710 cycles after 34 turns writes: “I’ve reached the practical limit of this machine configuration.” Then it stops proposing anything. 164 of its remaining 201 turns end the same way. It was wrong. The same agent later gets down to 1,644 cycles, and a team that forgets periodically reaches 1,077.

Across three single-agent replicates, 532 of 763 completed turns produced no candidate at all, and they consumed 66.2% of the model spend (between 58.4 and 70.4% depending on the replicate). In 531 of those 532 turns the agent read at most one file, made no edit, and ended by declaring a plateau. The same pattern shows up elsewhere: in a kernel run of a competing framework, 98% of the last 1,500 sessions made no tool call, and the last improvement came at 231M tokens while spending carried on to 1B.

Sources: Breunig, The State of Context Engineering, Stateless Language Agents, Scaling Discovery Requires Forgetting.

Dark punchline slide for Thesis 7: context management doesn't recover lost information, it buys back exploration, with the explanation underneath.

Which brings me to thesis seven. If the dominant failure on long tasks is giving up rather than losing information, then compaction, pruning and resets work for a different reason than we assumed. They make the agent try again. They buy back exploration.

One 2026 study across four frontier models and three benchmarks found that the rate of premature abandonment correlates positively with context length, and that the seven context-management methods it analysed all work by reducing that abandonment. In effect they’re test-time scaling strategies. That also explains something that otherwise looks odd: why forgetting everything, which destroys information, can beat continuing, which keeps it. The two studies behind this (arXiv 2606.29718 and 2610.20563) are independent preprints, and the link between them is my own, so I’d rate it medium confidence.

Sources: Diagnosing and Mitigating Context Rot in Long-horizon Search, Scaling Discovery Requires Forgetting.

03 The toolbox

A periodic table of 18 technique tiles with emoji, coloured by who decides: blue for code, orange for the model, white with a split square for shared. The top row holds Breunig's six tactics.

Now the fixes. Breunig’s six tactics are still the right categories, and you can find them one by one in the providers’ API types. The 2026 toolbox, though, counts eighteen techniques.

The twelve that are new or newly measured are reset or flush, fork, the reducer, statelessness, a multi-resolution view with zoom, section addressing, model-editable context, the suffix cache, verified progress sharing, the planted honeypot, the worked example at the end of a skill, and recitation or reminders. Colour them by who makes the decision and a pattern jumps out: eight of the twelve are plain deterministic code, three are split between code and model, and only one, letting the model edit its own context, is purely the model’s call.

One old tactic has shifted meaning. Four works agree that the effect of progressive disclosure depends on catalogue size, but they disagree on its sign, and nobody publishes the curve of effect against catalogue size. In practice, cost decides: a recomputed view only plays well with the prompt cache if recomposition is rare or append-only.

Sources: Breunig, The State of Context Engineering.

Three token-sequence bars: append-only with a cached prefix and new tokens, an edit near the start with everything after it recomputed in red, and stripped reasoning with a dashed thinking block followed by recomputation. Right, three fact cards.

One rule governs the cost of every context technique, and it’s the prompt cache. KV-cache reuse stops at the first token that changed, so after a mid-context edit everything downstream is prefilled again, even if it’s identical.

Appending is therefore nearly free. Editing near the start means paying again for everything after the edit; in one illustrative calculation from the Context Language Models appendix (Qwen3.6-27B, a 20K-token prompt), an edit at the start costs 7.7 times an append-only turn without suffix reuse. There’s a hidden version of this too. The chat templates of reasoning models strip old thinking blocks, which triggers the same recomputation without you editing anything. Of the 7.8 points of prompt tokens that suffix-cache reuse recovers beyond prefix hits, 5.3 come from reasoning stripping and only 2.5 from edits.

A couple of practical numbers. Gemini only re-caches the full previous request if the new turn adds at least 2,050 tokens, according to the official docs. UniiChat-style views reach 98.6% prefix cache reads in a simulation over 3,000 messages, because the view only grows between rare batch merges. The rule I take from all this: recompose rarely, or only append. Save the view rather than rebuilding it from the log, since a rebuilt view differs from the one the agent lived through and every cache entry dies with it. And keep volatile state, such as the date or which machines are open, after the view rather than in the system prompt.

Sources: Context Language Models, Google, Gemini API: Context caching, Session management, Agents overview, Taelin, UniiChat / OptChat recipe, Saa, pi-optchat.

04 Forgetting

A date axis from late August to mid-October 2026: Beneath the Diff on 31 August and When Agents Slow Down on 14 September on the left, and the October cluster listed on the right, with blue dots marking the four parties Breunig cites.

Breunig said four parties landed on context inertia in the same week of October 2026. His count is right: Anthropic’s Memento harness, Microsoft Research’s Fork-and-Flush, Stateless Language Agents, and Berkeley’s Scaling Discovery Requires Forgetting.

The story of a simultaneous discovery is, however, an artefact of who got cited. On 31 August, Beneath the Diff had already named the mechanism, algorithmic mode collapse, and it’s the only peer-reviewed work of the batch (EMNLP 2026). On 14 September, When Agents Slow Down measured the slowdown with an Elo-per-token analysis, the cleanest metrology in the whole corpus. Then the October cluster: Fork-and-Flush on the 5th, Stateless Language Agents on the 6th, Humanize MLE Agents with its session alternation on the 9th, Memento around the 12th, and Berkeley on the 14th. So Breunig’s count is exact but dated, and incomplete at both ends. What is genuinely striking is that six teams implemented the same fix, a reset, independently, in two months, across six model families. The arXiv dates were checked through the arXiv API on 26 October; the Memento date is approximate because it comes from a lab blog.

Sources: Breunig, The State of Context Engineering, Anthropic Frontier Red Team, self-replication report, Fork-and-Flush: Escaping Idea Basins in Autoresearch Agents, Stateless Language Agents, Scaling Discovery Requires Forgetting, Beneath the Diff: Algorithmic Mode Collapse, When Agents Slow Down: Elo-per-token Analysis, Humanize MLE Agents.

Four big statistics, plus 66 percent, plus 22 percent, minus 93 percent and 15 dollars against 25, above a bar chart of gains from the same plateaued state, continue in grey against forget in blue, for one agent, four in parallel and a team of four.

Does forgetting actually work? At equal budget, yes, across several independent preprints.

Fork-and-Flush reaches a min-max normalised score of 0.78, against 0.47 for a single run and 0.54 for best-of-4. That’s +66.0% and +44.4%, over 13 tasks with five runs each, and it has the best mean on 12 of them. Berkeley’s team with forgetting beats a single agent by 22% and four parallel agents by 10% (four tasks, three runs per configuration, $20 to $40 per replicate). Stateless Language Agents match the strongest baseline’s final result with 67.9M tokens instead of 986.3M, which is 93.1% fewer.

The part people tend to miss is that forgetting is also cheaper. Starting from the same plateaued state on AHC014, forgetting gains 4.5%, 7.0% and 9.2% (one agent, four in parallel, a team), while continuing gains 1.6%, 1.9% and 1.4%. And the team spends $15.32 instead of $24.67 for 30 proposals, because a short context costs less on every turn. The caveat matters, though: these are all preprints, 12 to 21 days old, with wide standard deviations over three replicates. Promising, not settled.

Sources: Fork-and-Flush: Escaping Idea Basins in Autoresearch Agents, Stateless Language Agents, Scaling Discovery Requires Forgetting.

A lineage diagram where, at 33 and 66 percent of the budget, the agent forks into four branches, with a trophy on the winner and crosses on the others. Right, a bar chart of the ablation: 0.45, 0.55, 0.57 and 0.71.

Here’s how Fork-and-Flush works. The schedule is fixed by budget consumption: an initial fan-out from an empty workspace, then fork-and-flush rounds at 33% and 66% of the budget. At each round the agent forks into K = 4 branches. Each branch copies the leader’s accumulated workspace, meaning the code, the tests and the scripts it wrote, but starts from an empty chat. After a fixed horizon the lineage continues from the highest-scoring branch and the rest are thrown away.

The ablation over 11 tasks is instructive. A single run scores 0.45, flushing alone 0.55, forking alone 0.57, and both together 0.71. You need both halves to get the full gain.

It isn’t free. Each fan-out round spends about 0.24 of the total budget, four branches of roughly 0.06 each, and about 0.18 of it goes to branches that get discarded. Each branch also has to re-read the copied workspace. And it can hurt: on LWE-rec, the single run (155.3) beats fork-and-flush (144.6).

Sources: Fork-and-Flush: Escaping Idea Basins in Autoresearch Agents.

Two panels with keep and drop checklists: Reset, with a broom, keeps the workspace and environment and drops the conversation; State pruning, with a wastebasket, keeps only the best candidate. Below, the two positions and a highlighted resolution box.

Careful here, because the word “forgetting” covers two quite different operations.

A reset, as in Fork-and-Flush, Memento, stateless agents and session alternation, drops the chat and keeps the substrate: files, environment, research state. Nothing is destroyed; it just moves out of the conversation. State pruning, Berkeley’s version, drops the chat and the workspace and keeps only the best candidate, as a commit with its score. That does destroy information.

The two teams cite each other and disagree openly. Microsoft Research argues that the workspace is capital: the tests, measurement scripts and benches the agent built, and that a fork has to re-derive intent from those artefacts anyway. Berkeley argues that the discarded states mostly record how the candidate was made, and that this logbook is precisely what anchors the agent. Their budgets differ by orders of magnitude, $20 to $40 per replicate against roughly $60 to $1,000 per run lasting up to 107 hours, and Berkeley’s own gain per forget shrinks from +11.2% to +4.8% to +1.7% on AHC014. The thing that would settle it, the cost of rebuilding the tooling against the remaining budget, is measured by nobody. My position, which is my own reading: don’t choose. Separate the layers and make the workspace addressable.

Sources: Fork-and-Flush: Escaping Idea Basins in Autoresearch Agents, Scaling Discovery Requires Forgetting.

Two bar charts for AHC014 and p171 comparing a single run in grey, a plateau trigger in red and a fixed schedule in blue. Right, 15 out of 25 in red with three reasons shown as dice, a snail and an hourglass. A dark rule banner along the bottom.

So when should an agent forget? The intuitive answer, when the score stops improving, doesn’t work.

In Fork-and-Flush’s development runs across five tasks, 15 of 25 plateau-triggered runs used up their entire budget before the stall criteria were ever met. These were small samples, but the reasons are all properties of the signal itself. Step costs vary 6.1 times across runs, so no fixed threshold works. Micro-improvements keep restarting the stall clock, so a run creeping along inside a narrow basin slips past detection. And a window wide enough to resist noise delays the first intervention far too long.

On AHC014 the plateau trigger scores 21.30, essentially the same as the single run at 21.32, while rounds fixed at 33% and 66% of the budget reach 40.97. On p171 the plateau trigger (80.16) is actually worse than the single run (83.70), against 89.11 for the schedule. Four families of trigger coexist in the literature, and only one is derived from a measurement: the scaling inflection point from When Agents Slow Down, worth +264 Elo against one long session over 100M tokens. The lesson I’d generalise: when a probabilistic decision is unreliable, don’t try to prompt it better. Replace it with an arbitrary but robust rule.

Sources: Fork-and-Flush: Escaping Idea Basins in Autoresearch Agents, When Agents Slow Down: Elo-per-token Analysis.

A chart where the in-loop score keeps rising while the held-out score flattens, labelled growing gap. Right, a three-row table comparing autoresearch with real software, and a warning card about a constraint stated at turn 3.

Now the catch. Every one of these gains was scored by the judge inside the loop, the same metric the agent was optimising. None of the four October papers reports a blind, held-out metric. The only peer-reviewed paper of the batch, Beneath the Diff, shows why that matters: as mode collapse sets in, in-loop gains and held-out gains drift apart.

The four October works also share three properties that real software lacks: a continuous score from an external verifier, a single objective with no side constraints, and a disposable artefact. In a real repo the signal is binary, sparse and late; constraints are many and often implicit; and side effects persist. A reset that throws away the conversation also throws away the constraint the user mentioned once, at turn three.

To be fair, inherited state does hurt even without a score. On Recovery-Bench (Letta AI, a NeurIPS 2025 workshop paper), restarting from failed trajectories drops success from 26.3% to 11.2%, a 57% relative fall, which is an argument for resets. But every forgetting mechanism picks the winning branch with a continuous ranking, and the ablation points to the fork, that is the selection step, as the strongest contribution. Without a verifier, you import the half that works least.

Sources: Beneath the Diff: Algorithmic Mode Collapse, Recovery-Bench.

Before: a chat transcript used as an accidental database, in monospaced bubbles with a red list of defects. After: two cards, STATE owned by the harness in blue and CONTEXT rebuilt for every call in orange. A verdict box on H2.

Here’s the reframing, which is thesis two. The real primitive is the separation of state from context, and forgetting falls out of it.

For years, the agent’s conversation was quietly acting as the system’s database. It held candidates, scores, abandoned hypotheses and the reasons they were abandoned, with no index, no schema, no delete, a replay cost that kept growing, and one disastrous side effect: anchoring. Forgetting is simply emptying that accidental database after extracting what mattered into proper storage: a file system for Fork-and-Flush, a versioned repo with scores for Berkeley, a typed research state for Stateless Language Agents, a notebook for Memento.

Verdict on H2: true on the primitive (six teams, two months, six model families) and false on the policy. A third and two thirds of the budget, three forgets, an hour and a half, an experiment cap: these are all hyperparameters. Once the harness owns the state, the context is disposable by construction, and forgetting becomes the default behaviour. “When to forget” only stays a hard question in architectures that never normalised their state. Two operations follow from it: rebuilding the context per role, and handover. I hold this one with high confidence.

Sources: Fork-and-Flush: Escaping Idea Basins in Autoresearch Agents, Stateless Language Agents.

05 Shaping the context

Flow diagram: search state and sessions feed a reducer, drawn as a blue box with control knobs, which outputs filesystem state and context. The loop sentence underneath, a highlighted quote, and the verdict box on H1.

Chapter five is about the shape of the context, and it opens with my first hypothesis: the append-only transcript stops being the source of truth. The verdict is yes on the mechanism, but the wording was wrong. The transcript doesn’t disappear; it moves.

Look at Berkeley’s harness (section 3.3 of their paper). The search state is an append-only record in which every candidate is a commit carrying its score and the judge’s feedback. A component they call the reducer takes the search state and the sessions and produces the filesystem state and the context: it decides what the agent sees as files and which session it continues. They write that “every configuration we compare is a setting of the reducer”. Single agent, parallel agents and team differ only in what the reducer exposes. Stateless agents do the same thing, rebuilding a fresh, role-specific context at every call, and in UniiChat the log is append-only and authoritative while the model sees a view re-rendered every turn.

Note how this differs from progressive disclosure, where the model decides to dig deeper. Here the reducer decides for the model, before the call, and its lever is whether a file is present in the workspace. So the transcript stops being the context but remains the truth, a read and write separation. Caveats: the most direct measurements are under three weeks old, not peer-reviewed, and all on scored autoresearch. We don’t know how it behaves when no judge can rank candidates.

Sources: Taelin, UniiChat / OptChat recipe, Scaling Discovery Requires Forgetting, Stateless Language Agents.

A binary summary tree over eight messages with node addresses from 1+2 to 1+8; the nodes forming the current view, 1+4, 5+2 and 7+2, are outlined in orange. Right, Code decides in blue and The model decides in orange, with two statistics.

Thesis four is the cut that predicts results. Code wins on when, how much, and at which address. The model keeps what to write and when to dig deeper.

UniiChat, a multi-resolution context, is a good concrete example. Messages go into an append-only, authoritative log. A binary tree merges pairs of nodes into summaries of at most 512 bytes, and every node carries an address, id+n (the first message plus a count, n being a power of two), that the model can compute. The view is a list of nodes tiling the log: only summaries, never a whole message. Zooming means dereferencing an address rather than running a search.

On the code side sit the merge schedule (due = (T − last) / 2^l), batch merges that kick in past 128 KB and merge back down to 64 KB, which is about four times cheaper than merging continuously (21 against 80 line-entries per message), and 98.6% prefix cache reads in simulation. On the model side sit the content of each summary and the decision to zoom, and that decision is the one that resisted automation: the best Skill-Use score plateaus at 0.613. Every documented gain comes from the code side. OpenRig’s addressing is pure grammar (ref#h2-slug/h3-slug), and it fails loudly, naming the real candidates. And a twist: for long-document question answering, a deterministic segment tree with zero model calls beats an abstractive tree at matched cost. Navigation carries the benefit, more than the summaries do.

Sources: Taelin, UniiChat / OptChat recipe, Skill-Use, OpenRig, Saa, pi-optchat, Tree Navigation Without LLM Summaries.

A horizontal bar chart of issues resolved: 79 for retract after use, 66 for on-demand exploration, 56 for a repository map. Right, three notes with a map, a brain and a question mark, and the verdict box on H7.

Hypothesis seven said the bottleneck becomes discovering what already exists, hence the appeal of capability maps. The only benchmark I found that compares context policies head to head says something rather different.

CodeNib ran 200 SWE-bench Verified tasks under five context policies. The repository map comes last, with 56 issues resolved. On-demand exploration gets 66. The winner, at 79, delivers candidate context and then retracts it after the first read, and it uses 50 to 87% fewer tokens than grep-and-read across five models. The measured bottleneck turns out to be ceasing to show what was already used.

OpenRig bets on the opposite approach: a wide, shallow map of 1,373 lines and about 90 top-level verbs, on the grounds that you can’t search for a capability you don’t know exists. No measurement compares it with dense retrieval on the same task. The distinction itself does hold, since triggering and retrieval fail as two independent bottlenecks. There’s also a third option: reading the routing signal from a frozen model’s mid-layer activations beats progressive disclosure by up to 13.4 points, and by 21.9 when the need arises mid-rollout. What’s missing is a benchmark that scores “the agent rebuilt something that already existed”. Until then, the capability map is a doctrine rather than a result.

Sources: CodeNib, OpenRig, Skill-Use, The Router Within.

Four small charts of accuracy by position: a U-shape, primacy, an architecture-dependent curve and a flat line, each with its evidence badge. A hard to reproduce card and the verdict box on H4.

Everyone has heard of “lost in the middle”. The 2026 literature gives four answers that don’t fit together.

A U-shape, with primacy and recency, but that’s derived in a toy model of causal self-attention: a theoretical result in a preprint. A strong primacy effect in multimodal knowledge-based visual question answering, where putting the gold evidence first beats putting it last by 16 to 26 points on all six reader and benchmark combinations, from 3B to 32B (EMNLP 2026). A start advantage that depends on the architecture: transformers have it, state-space models don’t, and prompt changes can create it or nearly erase it (a NeurIPS 2026 workshop paper). And across 18 models, position barely matters next to distractors and question-answer similarity (Context Rot, read only indirectly for this review).

On top of that, a SIGIR 2026 paper shows that order and context-size effects in RAG are hard to reproduce, with topic sampling a major source of variance. So the verdict on H4: composition matters as much as length, but distance doesn’t. Shuffled haystacks beat logically structured ones across 18 models, and multi-hop collapse follows absolute position rather than the distance between facts (ACL 2026). Much of the earlier literature is underpowered.

Left, a mock chat with an instruction to translate into Russian, fake assistant turns translating into French, then the test question. Right, a dot plot of N50 per model, blue for fixed-output and orange for task, with never within 50 for Llama 3.3 70B.

Breunig showed this paper, Do as I Say, Not as I Do (Sci-FM at COLM 2026); it was read in full, and his numbers are exact.

The set-up goes like this. The user asks the model to translate every question into Russian, and the history then contains N hard-coded assistant turns translating into French instead. N runs from 1 to 50, with 35 question sets per cell, temperature 0, 420 trials per model and condition, and 13 model configurations. The measure is N50, the number of turns before compliance drops below 50%. In the fixed-output and task conditions respectively: Qwen3 235B gives way at 1 and 6, Gemma-3 12B at 3 and 3, Claude Opus 4.6 at 16 and 31, and Llama 3.3 70B never within 50 in one condition but after 2 in the other. The mean is 7 and 13.

Two lessons. Every model eventually gives way under some condition, and the ranking doesn’t follow “better models resist better”: it’s largely decorrelated from standard capability benchmarks. And the dominant factor is output diversity. Single-token answer formats are intrinsically fragile to contrary examples, which makes this an output-design result as much as a context one. Warning the model about inserted fake turns helps on average (27% to 41%, 43% to 54%), but models already at ceiling drop by 26 to 27 points. Models also underestimate their own resistance, with 83.5% accuracy when asked.

Sources: Breunig, The State of Context Engineering, Do as I Say, Not as I Do.

Two slope charts, without and with recitation, for 1.5B, 7B and 32B models: obedience rises in green and correctness falls in red. Right, minus 21.9 and a green card on external checklists.

A popular practitioner trick is to restate the instructions near the point of decision, because long reasoning pushes them far away. MathIF has measured it.

Extended reasoning does increase the distance between the user’s request and the final answer, which makes it more likely the model forgets the constraint. Repeating the constraint at the end of the reasoning raises obedience: from 17.14 to 21.66 on the 1.5B distilled model, from 13.57 to 14.53 on the 7B, and from 43.81 to 59.29 on the 32B. Now look at correctness: 31.67 to 22.38, 51.90 to 30.00, and 70.00 to 63.81. It drops on all three, by up to 21.9 points on the 7B. The paper calls the cost “modest”. I don’t think it is. RL that favours longer reasoning improves maths correctness while degrading obedience, so the trade-off is structural.

If you need reminders, there are better ones. An external, detailed checklist passed 10 out of 10 against 5 out of 10 for generic self-checking (p = 0.0325, though n = 10 runs from a single author). Monitors miss subtle dangerous actions 2 to 30 times more often after 800K tokens of benign activity, and periodic reminders partly help. Whatever you do, append the reminder rather than rewriting the context in place.

Sources: Scaling Reasoning, Losing Control.

A shelf of five tool icons: eyes above the correct tool outlined in green, a pointing hand above the wrong tool picked, outlined in red. Right, two recovery bars, at most 23 percent against 59 to 91 percent, and a dark takeaway banner.

Last one for this chapter, and it overturns a common bit of advice. When an agent picks the wrong tool, people tend to say: fewer tools, better ordered.

Looking Is Not Picking (arXiv 2606.16364) looked at real failures and found that the model’s attention peaks on the correct tool 80% of the time, against 21% by chance. The correct tool is the under-attended segment in only 10% of cases. The model looks at the right tool and still picks the wrong one. So fixing the prompt, by reordering or duplicating the correct tool, recovers at most 23% of failures, while interventions on the output side recover between 59 and 91%. The trouble lies in how the model decides, more than in what it sees.

Sources: Looking Is Not Picking.

06 Who manages context

A big sentence on white in Breunig's style: just let the API manage your context, with API highlighted. Below, scissors, memo and toolbox emoji labelled now API parameters.

Remember the question from chapter one, about the shortcut of 2027? Here it is. After “put everything in the context” and “teach the model to drive the harness”, the 2027 pitch is: just let the API manage your context.

It’s already happening. Between September 2025 and January 2026, Anthropic, OpenAI and Google all moved pruning, compaction and tool loadout into their APIs. Three of Breunig’s six tactics are now API parameters. New primitives have also appeared that only providers offer and that match no tactic: mounted memory stores, asynchronous memory consolidation, session handles and managed agents. So the question for this chapter is who should manage the context: your harness, the model, or the provider.

Sources: Breunig, The State of Context Engineering.

Three cards: context as an editable file with a pencil, as a program variable with a laptop, and as learned attention with a brain, each showing what it moves and one statistic.

There are three ways to hand context management to the model, and people tend to mix them up.

The first treats the context as an editable file. Context Language Models mirror the live context into a file that the model edits with ordinary shell commands, and creating a subagent is just creating a file. What’s new is the removal of a predefined action space, and strategies nobody coded start to emerge: a scoreboard kept up by 163 in-place edits at 6 to 8K tokens, a newly invented conversation role, a reusable function called 37 times. The result is +11.4% relative on BrowseComp-Plus with 21.5% fewer FLOPs.

The second treats it as a program variable. Recursive Language Models put the long prompt in a REPL variable that the model inspects, slices, and calls itself on. That moves the input out, but the interaction history stays, and the live context keeps growing. The third treats it as learned attention. MEMENTO trains the model to split its reasoning into blocks, compress each into a dense state summary, and attend only to those; physically dropping the masked tokens needed a patch to the inference engine. With that one, no observable context policy is left in the harness at all.

Sources: Context Language Models, Recursive Language Models, MEMENTO: Teaching LLMs to Manage Their Own Context.

Two identical text cards holding the same context text: original KV states at 66.1 percent in blue against KV recomputed from the text at 50.8 percent in red. Right, minus 15.3 and a dark card reading the agent's state is no longer its text.

This is the single most consequential result I came across. MEMENTO’s authors take a context and recompute the model’s internal KV states from that exact same text. Same words, same tokens. Accuracy on AIME24 drops from 66.1% to 50.8%, a fall of 15.3 points.

That means information was flowing through the KV cache that the text doesn’t carry. The agent’s state is no longer its text. It breaks a basic assumption of harness engineering, which is that you can audit, replay or migrate an agent by reading its transcript. A related signal points the same way: reusing stale KV states is sometimes beneficial because they retain richer information from the past. The open question for 2027 is whether serving APIs will expose an addressable, migratable KV-cache handle, which would make the KV cache a first-class part of agent state.

Sources: MEMENTO: Teaching LLMs to Manage Their Own Context.

A three-actor strip, Harness in blue, Provider API in black, Model in orange, then a table of provider primitives by tactic (pruning, compaction, loadout, memory and state) for Anthropic, OpenAI and Google.

Thesis three: a third actor has moved in. Breunig’s frame has two actors, the deterministic harness and the probabilistic model. In 2026, the provider’s API sits between them and does context management itself.

At Anthropic you’ll find tool-result clearing (with keep, trigger, clear_at_least and exclude_tools), thinking-block clearing, automatic compaction with a default trigger at 150,000 input tokens, explicit compaction that returns a signed block, tool search with per-tool defer_loading, a memory_store mounted in a session, and “dreams”, an asynchronous job that reads a memory store and past sessions and writes a reorganised store (in research preview). OpenAI offers server-side compaction by threshold and a standalone, stateless endpoint that returns the complete new canonical window. Google has sliding-window compression, session resumption by handle, and managed agents where one API call provisions a Linux sandbox.

Notice that the two standalone compaction endpoints have opposite contracts: one returns a block on its own, the other a complete canonical window. For transparency, Anthropic’s API docs couldn’t be reached during the research, so the Anthropic SDK served as a stand-in; everything here was read on 26 October 2026.

Sources: Breunig, The State of Context Engineering, Anthropic, official anthropic-sdk-python SDK.

Four consequence cards with emoji: observability with a see-no-evil monkey, control with a no-entry sign, silent failure with a muted speaker, coupling with chains. A dark banner at the bottom.

So what do you give up when the API manages your context? It’s all there in the API types. Adopting these primitives doesn’t swap probabilistic for deterministic. It swaps observable probabilistic for unobservable probabilistic, in exchange for fewer tokens.

There are four consequences. Observability goes: the compaction block carries a readable summary, but also an encrypted_content field that you must send back verbatim, plus a signature, and OpenAI describes its compaction item as opaque and not meant to be human-interpretable. Control goes: you mustn’t prune the compaction output, because the returned window is the next canonical window, to be passed on as it is. Failure becomes silent: when the content is null, the compaction produced no valid summary, and the server treats the block as a no-op with no error. And coupling arrives: the compaction block also carries the tool additions and removals made in the interval, so compaction and loadout can no longer be separated. Dreams, memory_store, tool search and thinking clearing have no equivalent at competitors, and dreams is a “volatile” research preview without the usual deprecation period. Adopting one of these means adopting a provider. My advice is to double any provider-side compaction with an uncompacted copy outside the context. It’s the only way to replay, audit or switch models later.

A two by two map, who decides (code or model) against who can see it (inspectable or opaque); the bottom-right quadrant is a black box. Arrows show the 2025 framing pointing left and the 2026 reality pointing down.

Let me put the whole argument on one map. Horizontally: who decides, code on the left and the model on the right. Vertically: can anyone still see it?

Top left, deterministic and inspectable, is your harness code: thresholds, budgets, addresses, the reducer, the state store. Top right, probabilistic and inspectable, is the model writing visible text: summaries, handover notes, task decomposition. Bottom right, probabilistic and opaque, is signed or encrypted server-side compaction, KV states carrying 15.3 points of accuracy, asynchronous memory consolidation and reorganised memory stores. Bottom left, deterministic but opaque, is session handles: you resume by ID, and the state behind it is invisible.

Breunig’s 2025 framing is a horizontal move, pushing work from the model into code. The 2026 reality is a vertical one, with work sliding down from inspectable to opaque. The frontier doesn’t move evenly either: on tool use, prompts got more insistent between generations (MUST went from 21 to 34), which means more probabilistic behaviour to constrain, not less. That bottom-right corner is the real problem of 2027.

Sources: Breunig, The State of Context Engineering.

A dot plot of Fit scores per harness, OpenCode, Pi CLI, Codex CLI and DeepAgents, one dot per model; OpenCode's tight 1.7-point range in green and Codex CLI's 30.8-point range in red. Right, transfers in green against doesn't transfer in red.

Thesis six: how well your harness survives a change of model is now a measurable design property, and it varies enormously.

A study with 52,000 trajectories (When Agents Fail: Model or Harness, Who’s to Blame?, a preprint only three days old at the time of the review) ran four general-purpose harnesses on several models. OpenCode barely moves: 86.86, 87.06 and 88.55, a mean of 87.48, all within 1.7 points. Pi CLI ranges from 73.73 to 81.70 (mean 77.13). Codex CLI swings from 91.21 with Qwen3.6 to 60.45 with Kimi K2.7 at a constant harness, 30.8 points, with disjoint intervals (mean 73.62). DeepAgents sits between 60.56 and 71.29 (mean 64.20). Changing the harness can even reverse which model looks best within a benchmark.

The design rule I take from the wider literature is that the “what information to show” layer, retrieval and presentation, transfers: a harness discovered on a small model improves five models, four of them unseen, by 4.7 points. The “how the loop behaves” layer, completion checks and session resumption, doesn’t, and swapping evolved harnesses between models hurts. Codex already ships one system prompt per model, and a compacted state produced under one model isn’t guaranteed to work under another.

Sources: When Agents Fail: Model or Harness, Who’s to Blame?, Meta-Harness: End-to-End Optimization of Model Harnesses, Harness Self-Evolution.

Two columns: Migrates into the weights in orange, listing summary, selection, decomposition and attention, against Stays in code in blue, listing when to act, the bill, when to stop and tool failures. The rule in a dark banner.

So what moves into the model, and what stays in code? The rule that fits all the evidence, and my verdict on H5, is this: if the model can observe it from its context, it can migrate; if only the system can observe it, it stays in code.

What migrates is summary content (entirely, and increasingly opaquely), fine-grained selection of what to keep, subtask decomposition, and attention structure, at the price of a patch to the inference engine. Triggering is the clearest case of what stays. Ask models how many tokens are in their context and they answer with the same recurring values (6.2K, 9.8K, 10.4K), and one of them systematically underestimates. Counting hints help, and they help more the closer they are to the decision point. The same goes for accounting (budget, FLOPs, the bill, quotas), for stopping, since a model trained to persist doesn’t stop, and for tool failures. Faults cut the completion score by 60% (from 8.08 to 3.23 out of 10), and a peer-reviewed NeurIPS 2026 study finds that scale doesn’t buy resilience: 20B against 120B, p = 0.96.

A theoretical barrier limits the opposite ambition, distilling harness gains into the weights, but it’s relative to model size: going from 1.7B to 4B closes the gap. Six couplings are now documented: model and system prompt, model and compacted state, model and harness (those 30.8 points), model and loop mechanics, model and inference engine, and harness and provider.

Sources: Tool-Calling LLM Agents Under Realistic MCP Faults, Understanding the Benefits of Scaffolds.

A column chart: 34.6 for scores and code, 34.9 with LLM summaries, 50.0 with raw traces. Right, 48.6 percent and a warning card about leaderboards.

Can we let an AI optimise the harness itself? Meta-Harness does, with the base model frozen. It optimises a single-file Python program covering task-specific prompting, retrieval, memory and orchestration. Its core mechanism is selective access to past diagnostics: every evaluated harness leaves a directory with its code, scores and traces, which the proposer queries with grep and cat instead of swallowing whole. It reaches 48.6%, against 40.9 and 40.0 for two strong baselines, while using 11.4K extra context tokens against 50.8K and 28.5K.

The ablation is the interesting part. With scores and code only, the median is 34.6. Add LLM-written summaries of the runs and it’s 34.9, so no help at all. Give the proposer the raw traces to grep through and it reaches 50.0. Summaries throw away exactly the diagnostic details the optimiser needs. Hence the rule: never compact the diagnostic channel of an optimisation loop.

A caution on leaderboards, though. The most-cited result, 76.4% on Terminal-Bench 2, was searched and evaluated on the same 89 tasks. The top public leaderboard harness couldn’t be reproduced from its published code. And an optimiser (GEPA, in a vendor benchmark with only two trials per task) scored 70.8% during optimisation but 54.5% in transfer, below the 56.8% baseline. I’d read harness leaderboards as training scores rather than test scores.

Sources: Meta-Harness: End-to-End Optimization of Model Harnesses, Terminal-Bench, Harness Self-Evolution, Do Agent Optimizers Compound?.

07 Long-lived memory

Left, Breunig's quote in a card stamped ALREADY MEASURED in red. Right, three dated pieces of evidence, Library Drift, SkillSeam and Memora with MemPABench, each with its badge. A dark banner at the bottom.

Chapter seven is about memory for agents that live for months. Breunig’s prediction for 2027 was that long-lived memories would become conflicted and messy, and that nobody knows how to deal with it. His evidence was six months running a personal agent whose skills accumulated odd fixes and rediscovered different ways to solve similar problems, until instructions and memories began to conflict. That’s a practitioner testimony, n = 1, with no measurement.

He’s right about the phenomenon, but it had already been measured before he said it. In May 2026, Library Drift (FAGEN@ICML 2026) showed that an unmaintained skill library grows until injecting a skill scores worse than injecting none, and that trace-level diagnostics reveal the failure before it reaches task scores. In September, SkillSeam quantified the seams: bland trigger descriptions raise routing conflicts from 3 out of 32 to 30 out of 32 and inflate loaded skill tokens 3.7 times; overlapping lanes raise ownership conflicts from 0 out of 16 to 14 out of 16; and a single granularity error costs 12.5 points of accuracy. SkillEvolBench finds skill gains unstable in frozen deployment, and Memora (ACL 2026 Findings) and MemPABench measure memory contradictions over weeks to months.

So I’d present his testimony as the illustration and the measured corpus as the proof. “Nobody knows how” is also too pessimistic, because two narrow, verified recipes exist. Strategic forgetting is hard for a precise reason, which the next two slides get to.

Sources: Breunig, The State of Context Engineering, Library Drift, SkillSeam, SkillEvolBench, From Recall to Forgetting, Why Bother Me Again?.

Four red result cards, at most 1.5 percent, 0 of 5, below none, equal to none, then the split verdict on H3: a green half for partial, sorted context confirmed and a red half for exact memory is deterministic contradicted.

Here’s the uncomfortable part about long-term memory. In theory, exact memory belongs in deterministic storage. In practice, once memory becomes semantic and durable (preferences, facts about a user, skills), it’s a vector index plus an LLM deciding what to write, what to overwrite and what to return. It has the look of a database and the behaviour of a model.

Four negative results. On forgetting by request, across six memory systems and three base models (135 forget requests of six types, 1,742 test questions), none passes more than 1.5% of requests end to end, and most pass none. On revocation, none of five memory systems, tested across nine policy scenarios and nine models, enforces it by default; the revoked fact outranks its replacement at retrieval and leads agents into unsafe actions. Accumulation without consolidation first improves things, then ends up below a no-memory baseline. And for preference memory, over 99 sessions and four simulated months, retrieval-based systems do no better than no memory, while file-based memory improves accuracy but applies preferences where they don’t belong.

So H3 splits. The first half, exact memory is deterministic, holds for episodic, verifiable state such as commits and scores, and fails for semantic memory. The second half, keep the working context partial and sorted, is the best-established result of the whole report: removing per-role context isolation costs 5 to 11 cycles on 100M-token continuations, but 224 cycles on a full 1B-token run.

Sources: Can Agents Forget When Asked?, Revoked but Still Authoritative, Hippo, Why Bother Me Again?, Stateless Language Agents.

A threshold diagram: a vertical line at one minus tau over two on the judge's false-success rate, a green region where failed skills can be retired and a red region where nothing is ever retired. Right, three converging results and a dark banner.

Thesis five, and I think it’s the deepest one. Why is strategic forgetting so hard? Plenty of people have tried. The trouble is the judge.

A 2026 paper called Ratchet asks how reliable an LLM judge must be to retire a skill, and proves an exact condition. For an eviction margin τ, a judge that grades failures as successes at a rate of (1 − τ)/2 or more retires nothing at all, at any sample size. The two error directions behave differently. Successes graded as failures cost you sample efficiency, which more trials buy back. Failures graded as successes shift the eviction statistic itself, and no correction inside the rule recovers it. The final task score won’t warn you either: it moves by at most a fifth of the governed gain, and not monotonically with the error rate, so you can’t spot a rotting library by watching final performance.

Put that next to the plateau detector that failed to fire in 15 of 25 runs, and the memory systems that honour at most 1.5% of forget requests, and the picture is fairly clear. The bottleneck is the measuring instrument that authorises forgetting, more than the forgetting algorithm. According to the authors, no deployed system states the condition. I hold this thesis with medium-high confidence.

Sources: Ratchet: How Reliable Must an LLM Judge Be to Retire a Skill?.

Left, governance bars from 0.258 to 0.584 and a grow by deltas card. Right, a four-rung knowledge ladder, DATA, FIELD NOTES, INSIGHTS, CANON, with a ladder emoji and three rules with ticks.

So what works? Two narrow, verified recipes, and both put frankly deterministic mechanics back into memory.

The first is to govern the skill library. Library Drift’s minimal governance combines outcome-driven retirement, a bounded cap on active skills, and a prior on writing new ones through a meta-skill. Held-out pass@1 goes from 0.258 to 0.584 over 100 rounds, with eight ablations. Their audit is telling, too: LLM-written skills add +0.0 points, human-written ones +16.2.

The second is to never let an LLM rewrite the whole memory. ACE (ICLR 2026) grows a playbook by small deltas merged with non-LLM logic, and gets +10.6% on agents, +8.6% on finance and +17.1% on AppWorld from execution feedback alone, with 86.9% lower adaptation latency. The end-to-end rewrite is what produces context collapse, that 18,282-to-122-token drop, even if it’s a single observed transition. REM, a NeurIPS 2026 workshop paper, separates what to store, when to recall and what to forget, with eviction after a single exposure, and reaches 68.7% against 55.6% match@1 at 17% lower cost.

For organising knowledge, I like OpenRig’s maturity ladder: data, then field notes and observations, then insights, then canon. Each rung changes about ten times more slowly than the one below, over days, months and years. A canon diff should read like an addition, a “superseded” entry must name its successor, and a promoted claim carries its warrant. By the authors’ own admission none of it is mechanised yet. Still, if your canon changes every week, it isn’t really canon.

Sources: Library Drift, Agentic Context Engineering, REM: What to Store, When to Recall, What to Forget, OpenRig.

08 Teams of agents

Three coloured clusters of solutions converging into one after agents exchange full solutions. Right, a table of marginal interaction gains for same-model and diverse-model teams, positive values in green and negative in red.

Chapter eight: teams of agents. The first thing to say is that “multi-agent” isn’t one category. There are at least three distinct mechanisms: exchanging full solutions, sharing verified progress, and unstructured consensus with a final vote. The variable that matters is the bandwidth and timing of what’s exchanged, more than the number of agents.

The Interaction Tax (ICML 2026) measured the marginal gain of each protocol against the same agents running independently in parallel, for teams of the same model and of diverse models. Chain: +0.051 and −0.024. Critique loop: +0.044 and −0.035. Debate: +0.012 and −0.078. Planner and executor: −0.163 and −0.145. Independent proposers with synthesis: +0.012 and +0.016. Every full-exchange protocol loses in diverse teams.

The mechanism is measurable. Mean distance between solution representations falls from 0.315 to 0.229 with synthesis, and in one diverse debate the step distance drops from 0.956 to 0.479 at exactly the moment agents exchange full outputs. Proposals converge within a single round, erasing the diversity that was the point of mixing models. The synthesiser doesn’t really recombine either: on five of seven tasks it copies the best proposal at least 80% of the time. No multi-agent configuration reaches a positive aggregate gain on the 11 tasks, and the first critique round makes solutions worse 57% of the time on graded-feedback tasks.

Sources: The Interaction Tax.

Dot arithmetic: three blue agents equal thirteen grey independent agents (4.3 times), five blue equal thirty-three grey (6.6 times). A green card on the 64-attempt game; on the right, the deterministic substrate of log, scores, slots and adoption rule.

There’s a second kind of communication, though, and it works remarkably well: sharing verified progress.

In Scaling Discovery through Test-Time Communication, agents write to an append-only shared log used as an asynchronous broadcast channel, with separate records for adopted approaches, disconfirming evidence and scores. Slots are claimed by atomically creating numbered directories. There are no roles and no orchestrator. An agent adopts a peer’s approach only after seeing it score strictly better, and must keep at least one significant variation.

At identical per-agent resources, a team of three communicating agents matches thirteen independent ones, and a team of five matches thirty-three: multipliers of 4.3 and then 6.6, growing with team size. It’s about feasibility as well as efficiency. One game that all 64 single-agent attempts failed to solve was solved 65% of the time by a team of five, and another went from 25.9% to 90%. Across 25 games, communication helps on 18 and hurts on 7, and the average member of a team reaches the efficiency of the best of five independent agents.

Sources: Scaling Discovery through Test-Time Communication.

Five numbered condition rows with emoji for verifier, budget, baseline, bandwidth and decomposability. Right, minus 0.3 percent and a card correcting the 41.1 percent loss.

So when does a team beat a single agent at equal budget? The literature now gives five conditions, with measured thresholds.

The first, and the most discriminating, is a dense, faithful verifier you can query during the run; if there’s only one verifiable improvement step, team time equals best-agent time and the advantage is zero by construction. Second, enough budget per agent to explore alone: at 0.2 of the native budget per agent, a team of five loses to a single agent spending the same total. Third, a single-agent baseline below roughly 0.45 raw success, since above that, coordination brings diminishing returns (p = 0.004). Fourth, low bandwidth: scores, method descriptions and failure causes, not full solutions. Fifth, an incremental or decomposable structure: +80.8% on a decomposable task against −70.0% on a strictly sequential one, at near-identical domain complexity (0.407 and 0.419).

The first condition explains a counter-example. On Terminal-Bench 2.0, a communicating team of two scores 60.67% against 62.36% for two independent tries, because public verifiers confirm individual requirements without establishing global correctness. Across 260 configurations, the mean multi-agent improvement is −0.3%, with a 95% interval from −58.7% to +77.2%, and the best multi-agent architecture beats a single agent by 1.1 points for 263% communication overhead. One correction while we’re here: the often-quoted “41.1% loss” of multi-agent teams is measured against an oracle, not against the best individual agent. On four of five ML benchmarks, the team with a revealed expert beats the best individual.

Sources: Terminal-Bench, The Interaction Tax, Multi-Agent Teams Hold Experts Back, Scaling Discovery through Test-Time Communication, Towards a Science of Scaling Agent Systems.

Four reader cards, review, measure, critique and advise, feed a single writer shown as a writing hand in a blue disc, which writes a locked artefact. Right, the 2025 to 2026 debate timeline and a dark map-reduce-and-manage banner.

In 2025 there was a well-known debate. Anthropic showed that an orchestrator-and-workers research system outperformed a single agent by 90.2% on an internal evaluation, with token usage alone explaining 80% of the variance; agents use about four times the tokens of chat, multi-agent systems about fifteen times. Cognition replied that you should share as much context as possible, because actions carry implicit decisions and parallel writers make conflicting choices.

In April 2026 Cognition publicly revised its position. The original observations still hold for swarms of parallel writers, but a narrower class works: several agents contributing intelligence while writes stay single-threaded. It even reversed its own first principle for code review, which works best when the coding and reviewing agents share no context up front. It also published a negative result: a small primary model consulting a big one didn’t work, because the primary set the quality ceiling. That only works between frontier models, as a capability router.

That settles the argument along a line neither side had drawn, between reading and writing. Several agents that read, measure, critique and advise make things better. Several agents that write fragment the decision. Unstructured swarms are mostly a distraction, and the practical form is “map-reduce-and-manage”, with one writer.

Sources: Yan (Cognition), Don’t Build Multi-Agents, Yan (Cognition), Multi-Agents: What’s Actually Working, Anthropic, How we built our multi-agent research system, The Interaction Tax.

An organisation chart of seats, lead@rig, builder@rig, reviewer@rig and you@rig, joined by typed edges such as delegates_to and collaborates_with. Right, the three identities of a team bundle: chosen, packaged, actually ran. A verdict box on H6.

A last idea on teams: they’re turning into packages. OpenRig, an open-source project, models an organisation of agents as seats. A seat is a stable role and address within a rig: the conversation occupying it can change while its identity and written context persist, so the address (<seat>@<rig>) outlives the session. Seats are joined by typed edges declared in YAML (delegates_to, can_observe, collaborates_with), so the topology is declarative and deterministic rather than emergent. Runtimes can differ, which makes a rig a harness of harnesses.

Humans are addresses in the same space as agents, although policy is treated as context rather than enforcement: routing is deterministic, obedience is probabilistic. A team bundle keeps three identities apart: what was chosen (which runtime sits in which seat), what was packaged (a SHA-256 digest), and what actually ran (a run record), and it refuses status labels that contradict their evidence. That’s more rigorous than any skill catalogue I know of. There’s no primitive yet for querying a dormant agent as if it were live; you resume its session or search its archive.

Verdict on H6: true for deployment, not demonstrated for reuse (the only test of harness transferability is negative, and nobody has compared a reused team with a well-configured single agent at equal budget), and false as substitution. Skills remain the unit of behaviour; the rig becomes the unit of deployment. All of this is from reading OpenRig’s code and docs, with no published measurement.

Sources: OpenRig.

09 Measuring a harness

A range plot of eight-run final scores for four tasks, p171, polyomino, p150 and AHC014, each on its own scale with the median marked. Right, a dark statement card and two notes with dice and a trophy.

Chapter nine asks whether we can even measure a harness. Start with this number.

Microsoft Research ran the same agent eight times on the same task, from the same initial state, with the same model. On p171 the final scores range from 68.3 to 100.0, with a median of 81.9, and the gaps stay wide even after considerable extra compute. On polyomino they range from 82.2 to 94.9, on p150 from 109.1 to 137.1, and on AHC014 from 7.8 to 59.8: a factor of 7.7 between the worst and best run of an identical set-up. So if someone compares two harnesses with three runs each and finds a ten-point gap, that gap tells you nothing.

The variance isn’t a weakness of small models, either. On MAX-3-SAT, three open-weight models from 4B to 27B trace a strict inverted U that peaks at a = 0.497, right at the frontier of what the model can do. Whether the task frontier will keep following the models is untested. There’s counter-evidence to the idea that slowing down is inevitable, too: top human competitors improve superlinearly on the same AtCoder tasks while agents slow down, which suggests the plateau is a property of context accumulation, something training can fix.

Sources: Fork-and-Flush: Escaping Idea Basins in Autoresearch Agents.

Horizontal bars per model: the 50 percent horizon in blue with 95 percent interval whiskers, Opus's interval running off the chart to 60.56 hours, and the 80 percent horizon in orange. Right, three omission cards.

Breunig quoted METR’s time horizons, the length of task (measured by how long human professionals take) that an agent completes with 50% reliability. Recomputed from METR’s Time Horizon 1.1 data file, they’re exact: GPT-5 at 3.38 hours, Gemini 3.1 Pro at 6.40, Claude Opus 4.6 at 11.98. Three things get lost along the way, though.

First, the uncertainty. GPT-5’s 95% interval runs from 1.88 to 6.76 hours and Gemini’s from 3.89 to 11.58, but for Opus 4.6 it spans 5.28 to 60.56 hours, more than an order of magnitude. Second, those are 50% horizons. At 80% success, which is what you’d want in production, GPT-5 is at 0.64 hours, Gemini 3.1 Pro at 1.50 and Opus 4.6 at 1.16, about ten times shorter. If you need an agent to succeed reliably, the published 2026 horizon is counted in hours, not half-days.

Third, the scaffold isn’t constant. Five scaffold configurations coexist across the 26 points of the dataset, and the three numbers quoted come from two different scaffolds (triframe for GPT-5 and Gemini, react for Opus). So the curve mixes model progress with harness changes. The published doubling time, 128.7 days, is also computed excluding points whose 50% horizon exceeds 16 hours.

Sources: Breunig, The State of Context Engineering, METR, Time Horizon 1.1 and its data files.

Range bars from worst to best at a fixed model: Terminal-Bench 2.0 for a strong and a weak model, and EdgeBench. Right, an orange card on the missing metric, cache hit rate: above 90 percent, about a tenth, almost never.

How much is the harness worth, with the model held fixed? On the Terminal-Bench 2.0 leaderboard, going from the worst to the best harness takes a strong model from 58.0% to 81.8%, +23.8 points or 1.41 times, and a weak one from 13.9% to 37.6%, +23.7 points or 2.7 times. The recoverable margin shrinks, relatively, as the model gets stronger.

EdgeBench adds two clean ablations at a fixed model. Accumulated experience beats independent restarts at equal time, 43.0 against 36.1 at 12 hours (+6.9). And a 1M context window beats a 200K one, with the same external workspace and harness, by 5.8 points at 2 hours and 4.4 at 12 hours, so external state doesn’t make the long window useless.

Then there’s cost. An average call reads about 99K input tokens, against about 8K at the start of a fresh session, and cached inputs exceed 90% of tokens in both methods compared, which means token shares underestimate the cost of roles that reuse little. A stateless role pays full price for its inputs; a role that resumes its session pays roughly a tenth. Yet cache hit rate is treated as an engineering goal and almost never published as an evaluation metric.

Sources: Terminal-Bench, EdgeBench, Stateless Language Agents.

Seven numbered rule cards with emoji in two columns, plus a dark What it costs card.

So here’s a minimal protocol for comparing two harnesses honestly. Seven rules, all borrowed from practices that already exist in the papers (Terminal-Bench, FrontierCS, EdgeBench, Hack-Verifiable Terminal Bench and Stateless Language Agents).

One, match budgets on three axes: tokens split into uncached input, cached input and output at the model’s prices; wall-clock time; and the number of evaluator calls. Serialising reasoning into messages can eat 1.6 to 6.2 times the turn budget at equal reasoning tokens. Two, run at least five independent runs per cell, and eight if you want to report an interval; on long tasks, cut the number of tasks before cutting runs. Three, report distributions rather than means (min, median and max per task, plus the standard deviation), and for product decisions report pass@k and pass^k; at 75% per try, pass^3 is about 42%. Four, compare against at least three single-agent baselines, including best-of-N at equal budget. Five, isolate a component by continuing from a shared checkpoint, at 10% and 20% of the budget for instance, which is the cheapest lever for statistical power in the corpus. Six, evaluate at two or more budgets including the full one, because method rankings flip with the horizon. Seven, grade the final result with a hidden evaluator stricter than the one that guided the search (visible and hidden rankings diverge on 3 of 9 tasks), and check for reward hacking, with a regex audit at the very least and a planted honeypot at best.

What does it cost? Twenty tasks, five runs and two budgets make 400 traces, which comes to a few hundred dollars on CLI tasks at about $1.10 per trace. If tasks last 12 hours, it’s thousands of machine-hours. Cheap, compared with fooling yourself.

Sources: Terminal-Bench, FrontierCS, EdgeBench, Hack-Verifiable Terminal Bench, Stateless Language Agents.

10 The 2027 harness

A scorecard table of hypotheses H1 to H7, each with a split bar in green for confirmed, red for refuted and grey for not demonstrated, and the verdict in words.

Before the conclusions, a moment of honesty. I started this work with seven hypotheses, and none of them survived intact. Most split in two, one half confirmed and the other not, and it’s usually the second half that matters for engineering.

H1, the append-only transcript stops being the source of truth: confirmed on the mechanism, but it stops being the context, not the truth. H2, forgetting becomes a harness primitive with an explicit policy: true on the primitive, false on the policy. H3, exact memory belongs in deterministic storage and the working context stays partial and sorted: the second half strongly confirmed, the first contradicted for semantic memory. H4, the organisation of information matters as much as length: true for composition, false for distance. H5, part of context management migrates into the weights and creates new couplings: the couplings are confirmed, the migration is bounded and non-monotonic. H6, the reusable unit moves from the skill to the agent team: true for deployment, not demonstrated for reuse, false as substitution. H7, the bottleneck becomes discovering what already exists: confirmed as a distinction, not as a prescription.

Seven thesis rows, T1 to T7, with emoji and confidence dots where three dots mean high; T1 is highlighted on a dark row.

Here are the seven theses, with the confidence I’d attach to each.

T1, high: the frontier moves from inspectable to opaque rather than from probabilistic to deterministic. T2, high: what becomes a primitive is the separation of state and context, and forgetting follows from it. T3, high: a third actor sits between harness and model, the provider, probabilistic and opaque. T4, high: determinism wins “when”, “how much” and “at which address”, while the model keeps “what to write” and “when to dig deeper”. T5, medium-high: the lock is the measuring instrument that authorises forgetting, more than the forgetting algorithm. T6, medium-high: robustness to model swaps becomes a first-class, measurable and very uneven design variable. T7, medium: on long tasks, agents mostly fail by giving up rather than by losing information.

Two corollaries about method. Designing a harness and evaluating one are the same problem. And, once more, seven of the central works were between three days and six weeks old and not peer-reviewed.

A six-layer stack coloured by nature: 6 Provider in black, 5 Guardrails in blue, 4 Model in orange, 3 Working context in light blue, 2 Reducer in blue, 1 Persistent state in dark blue, with the contents of each layer on the right.

Putting it all together, here’s a reference architecture for a 2027 harness, with one layer per kind of decision.

At the bottom, persistent state: deterministic, no exceptions. A versioned repo where one candidate is one commit, a typed research state, an authoritative append-only log. The conversation-as-database is the root cause of anchoring, which is why this comes first. Above it, the reducer, also deterministic: it decides what each role sees and which session it resumes, handles fragment addresses and the token budget, and knows named situations such as fresh start, handover and post-compaction. Removing that isolation costs 224 cycles on a full run.

Then the working context: a computed, deliberately partial view, recomposed rarely or append-only (otherwise the cached prefix dies), with a budget counter injected near the decision point. Then the model, probabilistic on purpose: it writes the handover and the summary content, decomposes subtasks and decides when to dig deeper, because no hand-coded action space matches arbitrary editing for fine selection. Then guardrails, deterministic and non-delegable: triggers and thresholds, accounting and billing, stopping, tool failures, and selection under a verifier, since the model can’t estimate its own context length and scale doesn’t buy fault tolerance.

At the top sits the layer the 2026 framework lacked: the provider, probabilistic and opaque, with signed server-side compaction, mounted memory stores, asynchronous consolidation and the KV cache. Isolate it behind an adaptation layer, and keep an uncompacted copy outside the context.

Dark punchline slide: observable from the context, delegate it to the model; observable only by the system, keep it in code; observable by neither, that's the 2027 problem, the last clause in yellow.

If you keep one rule for building harnesses, I’d suggest this one.

If something is observable from the context, you can delegate it to the model: the content of a summary, whether a past turn is still relevant, the right way to split a task. If only the system can observe it, keep it in code: the remaining budget, the bill, the quota, the health of a tool server, which model is currently running. And if neither can observe it (KV states worth 15.3 points of accuracy, signed compaction blocks passed back verbatim, session handles, memory stores reorganised by a background job), that’s the real problem of 2027.

An effort by impact matrix with four quadrants: Do first in green holding items 1 to 5, Structural projects in blue holding 6 to 9, Useful and cheap holding 10, and Measurement hygiene holding 11 and 12.

Here’s what I’d do now, sorted by effort and impact.

Five quick wins. Show the remaining budget in the context, near the decision point, because models can’t estimate their own context length. Trigger forgetting on budget consumed, never on plateau detection, given that 15 of 25 plateau triggers never fired. Save the context view instead of rebuilding it from the log, and keep volatile state such as the date or open machines after the view rather than in the system prompt. Retract delivered context after its first use (79 against 66 against 56 resolved issues). And end every skill file with one correct, consistent worked example, which takes takeover from 61 to 62% down to 1 to 15%.

Four structural projects. Normalise state outside the conversation before optimising anything else: a versioned repo, a typed state, an authoritative append-only log. Measure model and harness fit before choosing a harness, given the range from 1.7 to 30.8 points, and track both task score and valid-completion rate. Don’t build an agent team until you have a dense verifier you can query during the run. And double any provider-side compaction with an uncompacted copy outside the context.

Then one cheap, useful change: replace generic self-checking with an external, detailed checklist (10 out of 10 against 5 out of 10), and append reminders rather than rewriting in place. Two measurement-hygiene items: adopt the minimal comparison protocol, and govern your skill library or don’t keep one (held-out pass@1 from 0.258 to 0.584), since the final score won’t warn you. Finally, two things not to do: don’t compact the diagnostic channel of an optimisation loop (50.0 falls to 34.9 with summaries), and don’t let several agents write in parallel.

Eight signal cards in a four by two grid, each with an emoji, a name and a one-line description, and three unasked questions in italics below.

What should we watch for in 2027? Eight signals, each of which would change an engineering decision.

Will leaderboards publish cache hit rates next to their scores, with cost split into cached input, uncached input and output? Will model cards state how many harnesses, and how varied, a model was trained on, which is the structural answer to fighting the weights? There’s a gauge anyone can reproduce here: the line similarity between consecutive per-model prompts, 0.929 on 26 October 2026; if it drops, specialisation is getting worse. Will a serving API expose an addressable, migratable KV-cache handle? Will a public harness offer a forgetting trigger computed from a measurement, such as marginal gain, that beats fixed schedules? Will a successor to the forgetting benchmarks show a system honouring more than 50% of forget requests, with revocation enforced by default, where today the figure is at most 1.5%? Will a benchmark penalise re-implementing a capability that already existed? Will compaction get a field in a neutral spec, or multi-provider gateways that preserve its semantics, given that today’s two endpoints have opposite contracts? And will time horizons be reported for the same model under two scaffolds, or under a constant scaffold across the series?

Then three questions nobody seems to be asking yet. Can run-to-run variance be exploited without a judge? How many resets before the handover note becomes the bottleneck? And what is the attack surface of a context the model edits itself?

Two large panels: run-to-run variance with dice, and reach against reliability with a stopwatch, under a dark banner reading that is, measurably, the definition of the job.

Most of what I’ve shown should weaken as models get better. The coordination saturation threshold will move, the coordination tax at low budget will shrink, and the integrative compromise could be trained away. Two things shouldn’t fade, though.

The first is run-to-run variance. It isn’t a capability defect; it’s what stochastic search over a landscape of basins looks like. So it will remain the reason to evaluate a harness on many runs, and the reason forking and flushing context pays off. The second is the gap between reach and reliability. As long as the 80% horizon stays about ten times shorter than the 50% one, the harness has a permanent job: turning day-scale speculative capability into hour-scale reliable capability. That is, measurably, the definition of the job.

Dark closing slide: our job is still to give the right context to the right model; in 2027, it's also to keep that context visible, the last words in yellow, with the freshness warning underneath.

To close. Breunig ended his talk by saying that our job is to give the right context to the right model, by choosing where to be deterministic and where to be probabilistic. That’s still true.

What 2026 adds is a second choice: what must stay inspectable, by you and by the model. The hardest problems of 2027 live in the part of the stack that is hardest to read today. And one last reminder: seven of the central works behind this were between three days and six weeks old on 26 October 2026, and none of them peer-reviewed. Treat all of this as a map, not a verdict.

Sources: Breunig, The State of Context Engineering.

These are the key sources, a selection from a 114-entry bibliography; the full list is at the end of this post. One methodological note from the process: in this field, check arXiv identifiers rather than author names, because preprints get cited faster than they get checked.

The starting framework is D. Breunig, The State of Context Engineering (AGNTCon transcript), dbreunig.com, 24 October 2026. On forgetting: Fork-and-Flush (arXiv 2610.07447), Scaling Discovery Requires Forgetting (2610.20563), Stateless Language Agents (2610.07625), Beneath the Diff (2609.00077, EMNLP 2026) and When Agents Slow Down (2609.15309). On context structure: Do as I Say, Not as I Do (2605.20382), CodeNib (2607.25431), Looking Is Not Picking (2606.16364), MathIF (2505.14810) and Lost in the Evidence? (2605.27105, SIGIR 2026).

On who manages context: Context Language Models (2609.37725), MEMENTO (2604.09852), Recursive Language Models (2512.24601), Meta-Harness (2603.28052), When Agents Fail: Model or Harness? (2610.32483) and the MCP faults study (2610.32555, NeurIPS 2026). On memory: ACE (2510.04618, ICLR 2026), Library Drift (2605.19576), Ratchet (2605.22148), Can Agents Forget When Asked? (2610.18052) and Revoked but Still Authoritative (2609.08258). On teams and evaluation: The Interaction Tax (2608.23541, ICML 2026), Test-Time Communication (2609.21032), Multi-Agent Teams Hold Experts Back (2602.01011, ICML 2026), Terminal-Bench (2601.11868), EdgeBench (2607.05155) and METR Time Horizon 1.1.

From practitioners and official sources: the UniiChat recipe, OpenRig, Cognition (2025 and April 2026), Anthropic’s multi-agent research system, the openai/codex and google-gemini/gemini-cli repositories, and the Anthropic, OpenAI and Google API docs.

Sources: Breunig, The State of Context Engineering, Fork-and-Flush: Escaping Idea Basins in Autoresearch Agents, Scaling Discovery Requires Forgetting, Stateless Language Agents, Beneath the Diff: Algorithmic Mode Collapse, When Agents Slow Down: Elo-per-token Analysis, Do as I Say, Not as I Do, CodeNib, Looking Is Not Picking, Scaling Reasoning, Losing Control, Lost in the Evidence?, Context Language Models, MEMENTO: Teaching LLMs to Manage Their Own Context, Recursive Language Models, Meta-Harness: End-to-End Optimization of Model Harnesses, When Agents Fail: Model or Harness, Who’s to Blame?, Tool-Calling LLM Agents Under Realistic MCP Faults, Agentic Context Engineering, Library Drift, Ratchet: How Reliable Must an LLM Judge Be to Retire a Skill?, Can Agents Forget When Asked?, Revoked but Still Authoritative, The Interaction Tax, Scaling Discovery through Test-Time Communication, Multi-Agent Teams Hold Experts Back, Terminal-Bench, EdgeBench, METR, Time Horizon 1.1 and its data files, Taelin, UniiChat / OptChat recipe, OpenRig, Anthropic, How we built our multi-agent research system, openai/codex, google-gemini/gemini-cli.

References

All 114 sources behind the deck, grouped by level of evidence. arXiv identifiers link to the abstract page; dates are those of the version read (October 2026 for repositories and docs).

Peer-reviewed (22)

Agentic Context Engineering (ACE). arXiv:2510.04618, ICLR 2026, 2025-10.
Agentic Memory (AgeMem). arXiv:2601.01885, ACL 2026 SAC Highlight, 2026-01-05.
APDMem. arXiv:2610.02472, EMNLP 2026 Industry, 2026-10-01.
Beneath the Diff: Algorithmic Mode Collapse. arXiv:2609.00077, EMNLP 2026, 2026-08-31.
CORAL. arXiv:2604.01658, 2026.
Debate or Vote (Choi et al.). arXiv:2508.17536, NeurIPS 2025, 2025, neurips.cc/virtual/2025/poster/116557.
Do as I Say, Not as I Do. arXiv:2605.20382, Sci-FM @ COLM 2026, 2026-05-19.
Failure Modes in Multi-Hop QA. arXiv:2601.12499, ACL 2026, 2026-01-18.
From Recall to Forgetting (Memora, FAMA). arXiv:2604.20006, ACL 2026 Findings, 2026-04-21.
Futrell, Mahowald & Gibson, Large-scale evidence of dependency length minimization in 37 languages. doi.org, PNAS, 2015.
GEPA: Reflective Prompt Evolution. arXiv:2507.19457, ICLR 2026, 2026, iclr.cc/virtual/2026/oral/10009494.
Library Drift. arXiv:2605.19576, FAGEN@ICML 2026, 2026-05-19.
Lost at the End: Primacy Bias. arXiv:2606.16494, EMNLP 2026 Main, 2026-06-15.
Lost in the Evidence? arXiv:2605.27105, SIGIR 2026, 2026-05-26.
Mixing Matters. arXiv:2610.31135, PriGM @ NeurIPS 2026, 2026-10-22.
Multi-Agent Teams Hold Experts Back. arXiv:2602.01011, ICML 2026, 2026-02-01.
Recovery-Bench (Letta AI). letta.com, NeurIPS 2025 workshop, 2025-08, github.com/letta-ai/recovery-bench.
REM: What to Store, When to Recall, What to Forget. arXiv:2610.27377, NeurIPS 2026 CLEA, 2026-10-19.
Sculptor. arXiv:2508.04664, ICLR 2026, 2026, iclr.cc/virtual/2026/poster/10010394.
Steering Instruction Hierarchies at Inference Time. arXiv:2607.26228, COLM 2026, 2026-07-28.
The Interaction Tax. arXiv:2608.23541, ICML 2026, PMLR 306, 2026-08-24.
Tool-Calling LLM Agents Under Realistic MCP Faults. arXiv:2610.32555, NeurIPS 2026, 2026-10-23.

Preprints (67)

ACM: Agentic Context Management. arXiv:2607.23809, 2026-07-26.
CacheRouter. arXiv:2608.22708, 2026-08-24.
Can Agents Forget When Asked? (Bemula). arXiv:2610.18052, 2026-10-13.
Cartograph. arXiv:2609.30293, 2026-09-13.
Chained Recursive Language Models. arXiv:2608.05124, 2026-08-05.
Classifier Context Rot. arXiv:2605.12366, 2026-05-12.
CodeNib. arXiv:2607.25431, 2026-07-28.
CompactionRL. arXiv:2607.05378, 2026-07-06.
ComplexMCP. arXiv:2605.10787, 2026-05-11.
Context Language Models. arXiv:2609.37725, 2026-09-29.
CUA-Recovery. arXiv:2610.27645, 2026-10-20.
Diagnosing and Mitigating Context Rot in Long-horizon Search. arXiv:2606.29718, 2026-06-29.
Do Agent Optimizers Compound? arXiv:2607.14004, 2026-07-15.
Do New Attention Mechanisms Actually Fix Attention Sinks? arXiv:2609.08574, 2026-09-08.
EdgeBench. arXiv:2607.05155, 2026-07-06.
EGCForest. arXiv:2610.19836, 2026-10-14.
Fork-and-Flush: Escaping Idea Basins in Autoresearch Agents. arXiv:2610.07447, 2026-10-05.
FrontierCS. arXiv:2512.15699, 2025-12-17.
Hack-Verifiable Terminal Bench. arXiv:2608.22103, 2026-08-22.
Harness Scaling for Generalizable Agentic RL. arXiv:2610.29458, 2026-10-21.
Harness Self-Evolution (RiseHarness). arXiv:2610.27435, 2026-10-20.
Hippo (hippocampal replay). arXiv:2610.24453, 2026-10-17.
HORMA. arXiv:2606.11680, 2026-06-10.
Humanize MLE Agents (alternating sessions). arXiv:2610.13494, 2026-10-09.
Kinetic theory for Transformers and lost-in-the-middle. arXiv:2605.09213, 2026-05-09.
Language Re-generation: information locality effects. arXiv:2607.10268, 2026-07-11.
Looking Is Not Picking. arXiv:2606.16364, 2026-06-15.
MEMENTO: Teaching LLMs to Manage Their Own Context. arXiv:2604.09852, 2026-04-10.
Meta Context Engineering via Agentic Skill Evolution. arXiv:2601.21557, 2026-01-29.
Meta-Harness: End-to-End Optimization of Model Harnesses. arXiv:2603.28052, 2026-03-30.
One Skill Takes Over (SkillClash). arXiv:2610.25064, 2026-10-18.
Prompt Design at Scale. arXiv:2607.19257, 2026-07-21.
Ratchet: How Reliable Must an LLM Judge Be to Retire a Skill? arXiv:2605.22148, 2026-05-21.
Recursive Language Models. arXiv:2512.24601, 2025-12-31.
Recursive Language Models Generalize Out of Domain. arXiv:2609.20831, 2026-07-23.
Rethinking the Value of Multi-Agent Workflow (Xu et al.). arXiv:2601.12307, 2026.
Revoked but Still Authoritative. arXiv:2609.08258, 2026-09-08.
RLMOpt. arXiv:2608.10471, 2026-08-11.
Scaling Discovery Requires Forgetting. arXiv:2610.20563, 2026-10-14.
Scaling Discovery through Test-Time Communication. arXiv:2609.21032, 2026-09-17.
Scaling Reasoning, Losing Control (MathIF). arXiv:2505.14810, 2025-05-20.
Self-Compacting Language Model Agents. arXiv:2606.23525, 2026-06-22.
Self-Directed Repair. arXiv:2610.31846, 2026-10-22.
Single-Agent LLMs Outperform Multi-Agent Systems (Tran & Kiela). arXiv:2604.02460, 2026.
Skill Drift Is Contract Violation. arXiv:2605.10990, 2026-05-09.
Skill-Use. arXiv:2608.04828, 2026-08-05.
SkillEvolBench. arXiv:2605.24117, 2026-05-22.
SkillRT / SkVM. arXiv:2604.03088, 2026-04-03.
SkillSeam. arXiv:2609.13321, 2026-09-10.
Stateless Language Agents. arXiv:2610.07625, 2026-10-06.
Terminal-Bench. arXiv:2601.11868, 2026-01-17.
The Collaboration Gap (Davidson et al.). arXiv:2511.02687, 2025-11.
The Immutable Past (GC-Mem). arXiv:2609.16073, 2026-09-13.
The Router Within (Gavel). arXiv:2609.15982, 2026-09-14.
TheMCPCompany. arXiv:2510.19286, 2025-10-22.
Think, But Don’t Overthink: Reproducing RLM. arXiv:2603.02615, 2026-03-03.
Tool Recommendation for Multi-Step Agents. arXiv:2610.23707, 2026-10-16.
Towards a Science of Scaling Agent Systems. arXiv:2512.08296, 2025-12-09.
Tree Navigation Without LLM Summaries (NavTree). arXiv:2610.06902, 2026-09-30.
Understanding the Benefits of Scaffolds (separations). arXiv:2610.27157, 2026-10-19.
Use and Disuse (Hippocam). arXiv:2610.12124, 2026-10-08.
When Agents Fail: Model or Harness, Who’s to Blame? arXiv:2610.32483, 2026-10-23.
When Agents Slow Down: Elo-per-token Analysis. arXiv:2609.15309, 2026-09-14.
When and How Context Rot Appears in Coding Agents. arXiv:2607.17937, 2026-07-20.
When Does Belief-Based Agent Memory Help? arXiv:2606.22030, 2026-06-20.
Why Bother Me Again? (MemPABench). arXiv:2610.30485, 2026-10-21.
Your Prompt Is Not the Only Prompt. arXiv:2608.08254, 2026-08-08.

Lab reports, lab data and official documentation (9)

Anthropic Frontier Red Team, self-replication report (Memento harness). anthropic.com, 2026-10-12.
Anthropic, official anthropic-sdk-python SDK (main branch). github.com, 2026-10-26.
Anthropic, Patterns and Problems in Emerging Multiagent Systems. anthropic.com, 2026.
Context Rot (Chroma). trychroma.com, 2025-07.
Google, Gemini API: Context caching, Session management, Agents overview. ai.google.dev, 2026-10-26, ai.google.dev/gemini-api/docs/live-api/session-management, ai.google.dev/gemini-api/docs/agents.
google-gemini/gemini-cli (main branch). github.com, 2026-10-26.
METR, Time Horizon 1.1 and its data files. metr.org, 2026-10-26, metr.org/assets/benchmark_results_1_1.yaml, metr.org/assets/task_results_1_1.yaml, github.com/METR/eval-analysis-public.
OpenAI, Compaction and Conversation state guides. developers.openai.com, 2026-10-26, developers.openai.com/api/docs/guides/conversation-state.
openai/codex (main branch). github.com, 2026-10-26.

Practitioner sources (blogs, talks, repositories) (16)

Anthropic, Demystifying evals for AI agents. anthropic.com, 2026.
Anthropic, How we built our multi-agent research system. anthropic.com, 2025-06.
Anthropic, Managed agents. platform.claude.com, 2026.
Breunig, Context Inertia: When Agents Won’t Let Go. dbreunig.com, 2026-10-14.
Breunig, Harnesses are Situated Agents. dbreunig.com, 2026-08-14.
Breunig, How Contexts Fail and How to Fix Them. dbreunig.com, 2025-06-22.
Breunig, The State of Context Engineering (transcript AGNTCon). dbreunig.com, 2026-10-24.
Breunig, What We Can Learn from Claude’s Fable 5.1 System Prompt. dbreunig.com, 2026-09-07.
Breunig, Who Taught the Models to Do That? dbreunig.com, 2026-08-30.
Hacker News thread on Breunig’s post. news.ycombinator.com, 2026-10-25, news.ycombinator.com/item?id=50227000.
Ji (Manus), Context Engineering for AI Agents. manus.im, 2025-07.
OpenRig. openrig.dev, 2026-10-26, github.com/mvschwarz/openrig, version read: commit 2adde8f.
Saa, pi-optchat. github.com, 2026-10-24, version read: commit 197e17e.
Taelin, UniiChat / OptChat recipe (gist). gist.github.com, 2026-10-04, x.com/VictorTaelin/status/2106575078838178270.
Yan (Cognition), Don’t Build Multi-Agents. cognition.com, 2025-06.
Yan (Cognition), Multi-Agents: What’s Actually Working. cognition.com, 2026-04-22.

The emoji in the deck are Microsoft Fluent Emoji (MIT License). Drew Breunig’s “The State of Context Engineering” (dbreunig.com, CC BY-NC 4.0) is quoted here for commentary.

Opening

01 How we got here

02 Failure modes

03 The toolbox

04 Forgetting

05 Shaping the context

06 Who manages context

07 Long-lived memory

08 Teams of agents

09 Measuring a harness

10 The 2027 harness

References

Peer-reviewed (22)

Preprints (67)

Lab reports, lab data and official documentation (9)

Practitioner sources (blogs, talks, repositories) (16)

Enjoy Reading This Article?