Context rot: why your long-running agent gets worse before it runs out of room

The natural assumption is that a long-running agent breaks when it fills its context window — you hit the token limit, the call fails, you handle the error. That’s not usually what happens. Agents degrade well before the window is full, and they don’t error when it happens. They just get worse: they re-read files they already read, forget a decision made forty turns ago, or act on a stale tool result instead of the corrected one three messages later. That’s context rot — model performance degrading as context grows, even with plenty of room left — and it’s a silent failure, which is what makes it dangerous in production.

Exhaustion is not the failure mode

Treat these as two separate problems, because the fixes are different:

• Context exhaustion — you hit the token limit. Loud, obvious, easy to catch with a counter and a threshold.
• Context rot — irrelevant tool outputs, superseded intermediate states, and redundant re-reads accumulate until the signal is buried in noise. The model doesn’t attend less to everything uniformly — it attends less accurately to the parts that matter, and there’s no error to catch. You find out from a wrong answer, not a stack trace.

A coding agent that read a file, then read it again after an edit, then read it a third time “to be sure,” now has three versions of that file in context and no explicit signal about which one is current. The model has to infer recency from position — and on a long enough transcript, it infers wrong.

Treat context like a managed cache, not a transcript

The teams getting reliable long-horizon agents stop treating context as an append-only log and start treating it as a budget with an eviction policy:

flowchart LR
  T[Tool call result] --> C{Still needed?}
  C -->|No, superseded| E[Evict / truncate]
  C -->|Yes, but bulky| S[Summarize in place]
  C -->|Yes, small| K[Keep as-is]
  S --> W[Working context]
  K --> W
  D[Durable decisions] --> M[(External memory)]

Three mechanisms do most of the work:

1. Clear and truncate at ingestion. Don’t wait until the window is nearly full to deal with a bloated tool result — truncate large outputs (file contents, API responses, search results) down to what’s relevant the moment they enter context, not as a later cleanup pass.
2. Compact on a trigger, not a deadline. When context crosses a threshold — a common default is around 70% of the window — summarize the older portion into a compact state and continue from there, rather than letting it ride to the limit and then scrambling.
3. Persist durable decisions outside the window. Facts the agent needs for the rest of the task (“the user’s account is on the legacy pricing tier,” “we already tried fix A, it failed”) shouldn’t depend on surviving a compaction pass. Write them to external memory — a scratchpad file, a task-state object — that’s re-injected explicitly, not left to chance inside a summary.

The failure mode compaction itself introduces

Compaction isn’t free. A summarization pass that isn’t scoped carefully can quietly drop constraints that mattered — a safety instruction, an approval requirement, a scope boundary the agent was given early in the session. If your compaction prompt just asks for “a summary of progress so far,” it will optimize for task continuity and treat governance constraints as disposable, because nothing marked them as different from any other line of context. Constraints that must survive compaction need to be tagged as such and re-asserted explicitly after every compaction pass — don’t assume a generic summarizer preserves them.

What to actually build

• Log context size and composition per turn (tool output vs. reasoning vs. history) so rot is visible before it produces a wrong answer, not after.
• Cap tool result size at the call site — most tools should return a reference or a bounded excerpt, not a full dump, with a follow-up call to fetch more if needed.
• Separate “working context” from “durable memory” as distinct storage, not two regions of the same window.
• Test agents on long-horizon tasks specifically — a 5-turn eval will never surface rot that only shows up at turn 40.

None of this is exotic engineering. It’s the same discipline you’d apply to any cache: know what’s in it, know why, and evict on purpose instead of by accident. We build context budgets — explicit truncation, triggered compaction, and external memory — into every long-running agent we ship, because “it has a big context window” is not the same claim as “it stays accurate for the length of the task.”