NeFut Logo NeFut
Admin Login

[CS.AI] What Context Does a Coding Agent Truly Require?

Published at: 2026-07-15 22:00 Last updated: 2026-07-17 08:46
#algorithm #optimization #C++

In modern coding agents, an agent can hold an entire repository in its context window. However, most of this reading is wasted. The key question is not how much context an agent can use, but what it actually needs. This paper investigates this question at the critical moment when the agent must edit code.

We separate the localization of the work site from the action, fixing the localization condition, varying how the code is represented, and scoring context against real issue resolution on SWE-bench Verified. The results indicate that the required context is starkly minimal. The signal resides mainly in the code being edited: natural language summaries answer almost none of the behavioral questions that the source code can answer ($4/45$ vs. $27/45$, held-out repositories, independent judges), and the gap lies in the representation, not the summarizer—a frontier model's summaries perform as poorly as a 3B model's.

Furthermore, the surrounding context hardly matters: across every multi-file instance in Verified, rendering the remainder of a file as UML skeletons and signatures resolves no more issues than outright deleting that remainder ($N=70$, exact McNemar $p=0.75$). This was our registered hypothesis, which ultimately failed.

Meanwhile, compressed context matches whole files at a third of the tokens: resolving an issue costs $19$K context tokens, not $94$K. Our instrument also yielded a notable finding: temperature-0 API inference flips about $9\text{perthousand}$ of per-instance outcomes, establishing a noise floor under every small effect reported on this benchmark, including ours. We release the instrument—gold-validated environments, per-instance proof that every reference edit is expressible from every arm's context, deterministic patch construction, and pre-registered hypotheses whose nulls we publish.

Blogger's Review: This paper empirically reveals that the context needed by coding agents when editing code is far less than expected, emphasizing that the signal within the code itself is more critical than the surrounding context. This finding offers vital guidance for the future development of coding tools and may reshape our traditional understanding of context handling.

Original Source: https://arxiv.org/abs/2607.09691

[h] Back to Home