Large language models (LLMs) are increasingly deployed as personalized assistants that interact with users over months or even years. As conversations become longer, feeding the full history to the model becomes computationally expensive and makes it hard for the model to consistently focus on the most relevant information for the current request. Structured memory systems have therefore been proposed to store and retrieve user‑specific context. In real‑world usage, users often seek practical advice such as recommendations, planning, or decision support rather than pure factual recall. Providing personalized guidance requires the model to integrate information across many past turns and to reason about evolving user preferences. Existing conversational memory benchmarks mainly evaluate retrieval accuracy and factual recall, overlooking this reasoning aspect. To address this gap we introduce the PRAGMA benchmark. PRAGMA contains curated longitudinal conversation histories, evidence annotations, and guidance scenarios grounded in evolving user contexts and incorrect user assumptions. Experiments across retrieval models, memory architectures, and long‑context models show that current systems struggle both to retrieve the appropriate conversational evidence and to leverage it for personalized guidance. These findings highlight the need for memory designs that support robust retrieval and memory‑grounded reasoning beyond simple evidence recall.
Review: PRAGMA offers the first systematic platform for assessing personalized reasoning in long‑term dialogues, exposing the shortcomings of existing memory solutions and pointing toward future memory‑reasoning integrated models.