Background
This study aims to evaluate whether retrieval-augmented generation (RAG) can serve as an efficient alternative for clinical reasoning over electronic health records (EHRs), specifically when dealing with long-context prompting.
Methods
We defined three EHR-based tasks that are replicable across health systems and vary in reasoning complexity:
- Extracting imaging procedures (modality, date, and anatomic site);
- Generating timelines of therapeutic antibiotic use;
- Identifying key diagnoses for hospitalization.
Using real inpatient clinical notes from a US academic health system, we evaluated three large language models (GPT-5.4-mini, Mistral Medium 3, DeepSeek V3.1) and compared their performance under varying amounts of provided context, particularly between targeted retrieval and using the most recent clinical notes.
Results
For the Imaging Procedures task, RAG strongly outperformed recent-note inputs, exceeding long-context performance (with F1 improvements ranging from 0.17-9.83) while using fewer than 8K tokens. Similar advantages were observed for Antibiotic Timelines.
Blogger's Review: This study reveals the immense potential of RAG in clinical reasoning, especially in handling complex medical records. By reducing the need for extensive context input, RAG not only improves reasoning accuracy but also enhances processing efficiency, presenting a promising outlook for clinical applications.