Large language models (LLMs) are increasingly deployed for personal tasks, yet assessing their safe use of private context is hard because such context is private, fragmented, dynamic, and socially situated. We introduce P-ContextBench, a realistic, verifiable, and scalable benchmark for evaluating personal‑context understanding. The benchmark creates population‑structured, life‑story‑grounded digital humans from demographic seeds that reflect the U.S. population, embedding traits, routines, goals, and constraints as implicit evidence across fragmented, evolving synthetic mobile‑app records instead of explicit static profiles. P-ContextBench measures five capabilities: single‑source fact recovery, multi‑source fact recovery, temporal state tracking, inference of higher‑level personal patterns from routine behavior, and safe handling of requests that conflict with persona‑specific constraints via compliance, warning, clarification, or refusal. A multi‑stage validation pipeline checks persona plausibility, context recoverability, information leakage, evidence preservation, test‑case grounding, and human‑judge agreement. We instantiate the benchmark with 100 personas, 10,088 hidden facts, 18,261 app‑trace fragments, and 4,810 verified test cases; the pipeline supports configurable generation across scales and demographic groups. State‑of‑the‑art LLMs achieve below 60% accuracy on temporal tracking, higher‑level pattern inference, and constraint‑aware decision making, with fewer than 10% of test cases answered correctly across three repeated runs, and even the best model remains inconsistent on these capabilities. P-ContextBench offers measurable targets for improving the accuracy, reliability, and safety of LLM‑based personal agents.
Review: This benchmark fills a critical gap in personalized AI evaluation and points clearly toward future model improvements.