Long‑horizon agentic tasks are pushing large language models (LLMs) beyond their context windows. To keep the task going, the agent must free up space, a process called context compaction. Compaction is inherently lossy: agents may silently lose task constraints, instructions, or the reasoning behind earlier decisions. Although compaction is a standard component of coding‑agent scaffolds, systematic studies with frontier models on realistic long‑horizon tasks are lacking.
We fix the OpenCode scaffold, implement five compaction strategies inside it, and evaluate three frontier models on three long‑horizon benchmarks: PaperBench, ProgramBench, and PostTrainBench. For the two 1 M‑context models, performance is essentially unchanged across compaction thresholds. The model with the smallest window, GPT‑5.3‑Codex, degrades at lower thresholds on PaperBench and shows a less clear drop on PostTrainBench. No method outperforms the default self‑summarization, and strategies that discard more content can penalize some models by up to 20 points while leaving others unaffected.
Compaction also silently removes task constraints: on ProgramBench the first summary drops about one‑third of the anti‑cheating rules on average, and the loss grows monotonically with more compactions. In real user‑agent sessions from SWE‑chat, instructions about how the agent should behave are dropped roughly six times more often than specifications of what to build. We argue that current benchmarks are compaction‑friendly: the objective is fixed and the agent’s progress is stored on disk, so most discarded information can be recovered except for constraints that appear nowhere else.
Review