In practice, LLM agents encounter situations where preferences, goals, and facts evolve over time. The model must retain older context while still using the most recent state in a conversation or log. If the model answers with an outdated value of the same variable, this is called a stale binding failure.
To measure when and why stale bindings occur, the authors introduce the Controlled In-Context Memory (CICM) benchmark. CICM tracks updates in dialogues and agent logs and checks whether the model correctly uses the latest information. Experiments reveal that even state‑of‑the‑art reasoning models often fail to recover the current state.
For open‑source models, probing can still retrieve the updated value, indicating that the model retains the new information internally but does not select it when generating an answer. Component tests on Qwen and Pythia families uncover an attention drift mechanism: during answer generation, attention disproportionately favors old values over the current one.
To explain this, the authors mathematically analyze a one‑layer Transformer. When attention scores are similar, the combined attention on several old values can exceed that on the current value, leading to stale outputs. Guided by this insight, they devise a training‑free intervention that redirects attention toward the current value at inference time.
Results show that, when the current value is directly requested, applying this per‑input intervention corrects most stale‑value errors across diverse model families while preserving nearly all originally correct answers. Thus, reliable context management requires more than memorizing updates—it demands using them to steer the model’s responses.
Review