MedLoCoMo is a Medical Long-Context Memory benchmark aimed at patient-specific clinical reasoning over multi-admission medical dialogue. Existing medical QA benchmarks primarily test short-context knowledge or single-document grounding, leaving open whether large language models (LLMs) can effectively utilize, connect, and infer from longitudinal patient histories.
We constructed MedLoCoMo from deidentified MIMIC-IV and MIMIC-IV-Note records by building admission-level clinical packets, synthesizing evidence-based doctor-patient conversations, and generating evidence-linked QA items across single-admission, cross-admission, and adversarial unanswerable settings. The benchmark includes 100 patient timelines averaging 1,669.8 turns, 29.7 sessions, and 74,512.2 tokens per conversation.
Across the evaluated baselines, cross-admission reasoning consistently proves harder than localized evidence usage, even when models have long context windows or utilize external memory or retrieval methods. The code and MedLoCoMo benchmark release is available at GitHub for use and reproducibility.
Blogger's Review: The introduction of the MedLoCoMo benchmark addresses a significant gap in the medical dialogue domain, especially regarding the complexities of managing long-term patient histories. This innovation presents new challenges and opportunities for LLMs in clinical reasoning, making it a worthy area for further exploration by researchers.