Context rot

Context rot is the measured decline in a model's ability to recall and use the information it has been given as its context window fills. A long transcript, an oversized retrieval payload, and a sprawling system prompt do not merely cost more to process. They make the answer worse, and they do so unevenly, because the model does not attend to every position in the window with equal fidelity.

The term was coined in mid-2025 by Chroma's research on long-context performance and became standard vocabulary in agent design documentation through early 2026. It matters because the industry spent the preceding years marketing ever-larger windows as a substitute for retrieval discipline. In customer support, where a session can run for dozens of turns while pulling in policy documents, order history, and tool output along the way, context rot is why an agent that handled the first ten questions cleanly starts contradicting something it was told much earlier in the same conversation.

What context rot describes

Context rot is a behavioral pattern rather than a bug in any one model, and three symptoms recur reliably enough to design around.

Positional unreliability. Information in the middle of a long context is recalled less reliably than information at the very beginning or the very end. A policy clause buried deep in a long retrieved set is functionally less present than the same clause at the top of the prompt, even though both are inside the window.

Distractor sensitivity. Accuracy falls when the context contains plausible but incorrect neighbors of the right answer, even when the right answer is present. Retrieval that returns a stack of similar refund policies for different regions hands the model a chance to pick the wrong one that a tighter result set would not have offered. Raising top-k in a RAG pipeline is therefore not a free improvement.

Early onset. Degradation begins well before the advertised token limit. The published window size is a hard ceiling on what the model will accept, not a promise of uniform performance up to that ceiling, and treating the two as the same thing is the design error the concept was named to correct.

Why a larger context window does not solve it

The intuitive fix is to buy more window, which helps with the narrow failure of hard truncation, where the oldest turns are simply dropped, and does very little for anything else. A larger window raises the ceiling without changing the attention economics underneath it. Signal does not get stronger as the window grows; noise gets more abundant. A model given the right paragraph and nothing else will use it, and the same model given that paragraph buried in pages of adjacent material may not.

The consequence is that window size and answer quality are separate axes, and the second is governed by context engineering rather than by procurement. Teams that treat a bigger window as license to stop curating the prompt end up with slower, costlier agents that are also less accurate.

How context rot shows up in long support conversations

Support is an unusually good place to watch this happen, because a session accumulates context from several uncoordinated sources at once. Consider a conversation that has run for ninety turns. The system prompt carries brand voice, escalation rules, and refund limits. Retrieved knowledge carries help center articles, some relevant forty turns ago and dead weight now. Tool results carry raw order payloads, subscription state, and a carrier response, each verbose and each still sitting in the window. The customer's own words are a small minority of the prompt by now, and all of it competes for one finite budget inside a single multi-turn conversation.

The failures are specific. The agent re-asks for an order number the customer already gave. It quotes a policy that a later clarification superseded. It contradicts an authorization decision it made itself. None of these look like retrieval failures in a trace, because the correct information is sitting in the prompt. They look like the agent ignoring what it was told, which is why teams often misdiagnose the problem as a grounding failure and tune retrieval instead of trimming the window.

How context rot is measured

Measuring context rot means measuring recall as a function of context length and position, not overall accuracy. The basic experiment plants a fact in a long context, asks a question that requires it, and varies both the volume of surrounding material and the depth at which the fact sits. The resulting surface shows where a given model and prompt design begin to fail, not merely whether they do.

Production measurement is harder and more useful. Instrument conversations by turn depth and prompt size, then score outcomes at each band with an eval suite built from real transcripts rather than synthetic needles. If resolution quality is flat through the early turns and falls off after, that drop-off point is a design constraint, and it should dictate when the agent compacts, escalates, or hands off rather than being reported as a single accuracy number for the agent as a whole.

Mitigating context rot in production

Every effective mitigation reduces to one principle: put less in the window and make what remains more relevant. Four techniques do most of the work.

Retrieve instead of stuffing. Pull the specific passages this turn needs rather than preloading the knowledge base and hoping the model finds the right part. Tighter top-k with reranking raises the ratio of useful to distracting content, which is the quantity that governs accuracy.

Compact the history. Periodically summarize resolved portions of the transcript into a structured block and drop the verbatim turns behind it. This technique, context compaction, is the standard response to rot in long-running sessions.

Clear stale tool output and definitions. Tool results that have been acted on can be reduced to their conclusion, and schemas for tools this conversation will never touch do not belong in the prompt at all. That second problem has its own name, MCP context bloat, and it is often the largest single contributor to a bloated window before the customer has said anything.

Scope work to sub-agents. Delegating a bounded task to a fresh agent with its own clean window keeps the parent conversation short. A multi-agent system pays a coordination cost for this, but it caps the size of any single context instead of letting one grow without limit.