Agentic memory enables large language model (LLM) agents to reuse past experiences, yet retrieved memories can distort inference even when they are benign, correctly stored, and appropriately retrieved. We call this failure mode memory over‑reliance.
Across several benchmarks and memory architectures we observe that memory is beneficial when past experience fully transfers to the current task, but becomes misleading when only a subset of the evidence transfers. The problem is strongest when the query and the stored memory overlap only partially, a pattern confirmed by controlled experiments that vary the amount of overlapping evidence.
Motivated by this finding we propose MEMTRIM, a plug‑and‑play framework that indexes memory evidence at write time and regulates its reuse at read time. MEMTRIM removes repeated or conflicting evidence while preserving useful memory‑specific information, requires no retraining, and works with both embedding‑based and structured memory systems.
Experiments demonstrate that MEMTRIM reduces memory over‑reliance while retaining the benefits of useful memory across models and memory settings.
Review