In agentic AI, prospective memory poses a significant challenge: the ability to execute an intention at a specific future cue or state while other activities are ongoing. We introduce PM-Bench, a text-based benchmark for measuring the prospective memory capabilities of modern Large Language Model (LLM) agents. Inspired by the Virtual Week paradigm from cognitive science, PM-Bench evaluates how well LLM agents maintain user intentions, execute delayed intentions, and monitor latent environmental changes.
Over the course of a simulated seven-day week, agents must continue an ongoing activity while deciding whether any deferred tasks are due. We compare eight state-of-the-art LLMs on PM-Bench under eight different agent configurations. PM-Bench proves challenging across all settings: the best method, a GPT-5.4 agent, achieves only a 65.1% F1 score in our evaluation. Moreover, no single strategy for improving prospective memory dominates across models. We release PM-Bench as a controlled testbed for diagnosing these failures and developing training or inference-time interventions that support reliable prospective behavior.
Blogger's Review: The introduction of PM-Bench provides a crucial tool for assessing the prospective memory capabilities of large language models, which is vital for the advancement of agentic AI. While current model performance leaves room for improvement, the establishment of this benchmark will drive future research, helping us better understand and enhance the memory capabilities of agents.