Abstract
LLM-based autonomous agents require external memory to overcome their statelessness and limited context window for long-term interaction and dynamic knowledge reasoning. However, existing memory retrieval methods often lack adaptability and sample efficiency, struggling to retrieve the right mixture of memories from heterogeneous stores.
We propose the Exploratory-Assimilating Reflection (EAR) framework for high initial retrieval performance and sample-efficient adaptation. EAR combines two mechanisms: Exploratory Reflection, which performs iterative search to bootstrap retrieval and collect useful experiences for each query; and Assimilating Reflection, which replays these experiences from an Experience Buffer to refine a global reranker more efficiently than methods relying only on immediate rewards.
Experiments show that EAR improves retrieval by up to 17.9% over the baseline retriever on two long-term dialogue benchmarks. We also demonstrate that EAR is highly sample-efficient and robust to noisy feedback.
Blogger's Review: The EAR framework effectively enhances long-term memory retrieval capabilities by introducing both exploratory and assimilative reflection mechanisms. Its sample efficiency and robustness to noise provide crucial insights for future LLM applications in dynamic interaction environments.