Large language models increasingly rely on external memory to handle tasks that require long‑horizon histories such as multi‑session dialogues and lengthy documents. Existing memory approaches typically optimize retrieval and memory construction separately, lacking a shared reward space to evaluate search, write, and stop decisions. To address this gap we introduce All‑in‑One Memory (AiO‑Mem), which treats these operations as transitions over a shared evidence state and directly assesses the semantic quality of each action within a unified reward space. AiO‑Mem compiles heterogeneous histories into source‑traceable multi‑view events, providing a common substrate for both retrieval and memory writing. A unified memory policy actor then rolls out complete interaction trajectories on this evidence state. A trajectory‑conditioned process reward model is trained via evidence‑grounded supervised fine‑tuning and trajectory‑level DPO to assign scores to the collected trajectories. These scores are aggregated into an action‑balanced trajectory reward and used by Process‑Aligned DAPO (PA‑DAPO) to optimize the memory policy. Across long‑context memory benchmarks, AiO‑Mem consistently improves answer accuracy over static memory systems and RL‑trained memory agents.
Review: The paper presents a compelling unified reward framework that tightly couples retrieval, writing, and stopping, offering a principled end‑to‑end optimization for memory agents and demonstrating solid gains on realistic long‑text tasks.