Long‑term memory has become a crucial component of large language model (LLM) agents, with most research focusing on accurate and efficient retrieval of relevant information. Yet a dependable memory system must also be able to forget sensitive user data upon request. To evaluate this capability we introduce MemCut, a systematic benchmark for user‑requested forgetting. The benchmark contains 456 hand‑designed secrets embedded in long‑horizon conversations, each fully annotated with an evidence trail to test recovery after deletion, including from messages that never state the secret directly. We define two recovery metrics: native recovery measures a reader’s success through the system’s retrieval, while residual recovery measures the same reader’s success when directly given surviving objects traced to disclosed evidence. We establish three reference deletion baselines across five memory designs and four LLMs to test how existing retrievers and write models support forgetting. Results show that disclosure removal reduces native recovery far more than residual recovery. LLM‑guided deletion of ten additional turns from twenty retrieval candidates leaves residual recovery between 39% and 76%. Subset sweeps over 144 facts, three systems, and two models reveal that widening the candidate pool at a fixed budget reduces residual recovery more than deleting all original candidates, identifying candidate coverage as a bottleneck in the reference baselines.
Review: MemCut offers a detailed experimental framework for assessing the forgetfulness of LLM memory systems, exposing current retrieval and writing models’ limitations in truly erasing sensitive information and pointing toward future secure memory mechanisms.