Context compaction

Context compaction is the practice of periodically summarizing an agent's accumulated conversation history and tool results into a compact, structured block, so a long-running session stays affordable and accurate instead of dragging the entire transcript into every model call. The block replaces the raw turns it covers, and the agent works from a shorter prompt that still carries the state it needs.

Compaction shipped quietly in agent harnesses in late 2025 and was named and generalized through the first half of 2026, with Anthropic's context engineering material presenting it alongside memory and tool clearing. The reason is arithmetic. A conversation that replays its full history every turn grows quadratically in cost and degrades in quality as the window fills. Compaction is how an agent handles a ninety-turn support conversation without either outcome.

What compaction does to a conversation history

A raw agent transcript is a chronological log of user turns, assistant turns, tool calls, and tool responses. Most of that log is settled. The customer's identity was verified early on. A shipping lookup returned a payload of which one field mattered.

Compaction rewrites the settled portion into a dense state description and discards the originals. Instead of dozens of turns of dialogue and several verbose tool call results, the prompt carries a block stating who the customer is, what has been verified, which sub-tasks are resolved and how, what commitments the agent has made, and what remains open. Recent turns stay verbatim, because recency is where nuance lives. This differs from ordinary conversation summarization in purpose: a summary is written for a human to read afterward, while a compaction block is written for a model to act on mid-session, so it is structured, it preserves identifiers exactly, and it is explicit about state rather than narrative.

When compaction triggers

Trigger policy is the main design decision, and the two common approaches fail differently.

By token threshold. Compact when the assembled prompt crosses a fraction of the available context window. This is the more principled trigger, because it responds to actual pressure rather than a proxy for it.

By turn count. Compact every fixed number of turns. Simpler to test, and wrong whenever turn length varies, which in support is always.

Mature implementations combine the two, with a token threshold as the real trigger and a turn ceiling as a backstop, and they avoid compacting mid-task. Compacting in the middle of a multi-step action is how authorization state gets dropped between the check and the action that depended on it, and waiting for a natural boundary removes that whole class of bug.

What survives compaction and what does not

The value of compaction depends almost entirely on the preservation rule, and a general instruction to summarize the conversation will eventually summarize away something load-bearing.

Preserve verbatim. The current objective, unresolved commitments the agent has made to the customer, customer and case identifiers, and authentication and authorization state, all of them values where paraphrase is worthless or dangerous. A reworded order number is useless, and an agent that recalls the customer was verified but not what for has lost the part that mattered.

Safe to summarize. Resolved sub-tasks and their outcomes, verbose tool output reduced to the field the agent used, and exploratory turns that led nowhere. A long carrier response compacts to a delivery date and a status.

Dropped outright. Stale retrieved passages tied to a closed question, and tool schemas for tools this session will not call. That second category overlaps with MCP context bloat, and clearing it is often a larger win than compacting dialogue, because tool definitions are fixed overhead that recurs on every single turn.

The risks of lossy summarization

Compaction is lossy, and its failures are quiet. The first risk is dropping something that mattered: a constraint mentioned in passing, an exception the agent granted, a detail that only becomes relevant later. The agent does not know the information is gone and behaves as though it never existed, which reads to the customer as a contradiction with no explanation. From outside, this is indistinguishable from context rot, and telling them apart requires keeping the pre-compaction transcript in a trace even after it leaves the prompt.

The second risk is compounding. A session long enough to compact several times produces summaries of summaries, and each pass introduces drift: details get vaguer and specifics give way to generalities that are technically true and operationally useless. The defense is to compact from the original transcript held outside the prompt rather than from the previous compaction block, so error does not accumulate across passes.

The third risk is losing authorization state mid-session, where the consequence is not merely a bad answer. An agent that forgets a verification step may redo it, which is annoying, or proceed as though it had happened, which is an incident. Authorization facts belong in structured fields the compactor may not rewrite, enforced by guardrails outside the model rather than by the model's memory of its own earlier decision.

How compaction interacts with cost and latency

Compaction trades a predictable small cost for an unbounded large one. Each pass is itself a model call over a substantial input, so compacting too often can cost more than it saves. Compacting at sensible intervals is still far cheaper than replaying a growing transcript every turn, because the transcript cost is paid repeatedly and the compaction cost once per pass.

The latency picture is more interesting. Prefill scales with input length, so a shorter prompt directly improves time to first token, the number a customer actually perceives in chat and the one that decides whether a voice agent sounds alive. Compaction is one of the few techniques that improves cost, quality, and perceived latency at once. The catch is that the compaction call is itself a latency spike wherever it lands, which argues for triggering it at task boundaries or running it between turns rather than in the critical path of a waiting customer.

Caching is the last interaction to plan for. Prompt caching rewards a stable prefix, and every compaction pass invalidates that prefix by rewriting the middle of the prompt. Compacting rarely and at predictable points preserves more cache hits than trimming continuously.