In Large Language Model (LLM) agents, the reliance on external memory systems to accumulate experience across tasks is increasing. However, nearly all existing approaches depend on fixed, hand-designed heuristics for memory access. We argue that this static view of memory is a core bottleneck for agentic learning because optimal memory behavior is fundamentally context-dependent.
In the early stages of tasks, minimal retrieval is beneficial due to sparse memory; recurring goal types benefit from plan reuse rather than generic nearest-neighbor lookup; stuck agents benefit from re-retrieval with alternative queries; and across long task streams, the memory store itself must be consolidated and pruned to remain useful.
We present Memory as a Controlled Process (MemCon), a framework that models memory operations as a Markov Decision Process and learns an online policy that adaptively decides when, what, and how much to retrieve, when to inject a distilled plan, and when to consolidate or forget. MemCon is backend-agnostic: it wraps any existing memory implementation, learns from task-by-task binary feedback with no pretraining and no additional LLM calls, and uses a lightweight tabular contextual bandit with UCB exploration that converges within tens of tasks.
Across 6 benchmarks, 3 agent frameworks, and 3 LLM backbones, MemCon consistently outperforms multiple memory baselines by up to 15.2 points in task success while reducing token consumption by 5-20%.
Blogger's Review: The MemCon framework enhances LLM agent efficiency by dynamically adjusting memory management strategies. This approach not only reduces resource consumption but also improves learning outcomes, showcasing the significance and potential of memory management in intelligent systems. Future research could further explore context adaptability in memory, advancing the progress of intelligent agents.