UMLS has become a knowledge base for millions of clinicians and health‑system developers worldwide. Researchers are increasingly leveraging knowledge graphs derived from UMLS to improve factual accuracy and reasoning traceability of language models in high‑risk tasks. Medical Question Answering (MQA) serves as the primary testbed for this integration, with benchmarks typically requiring models to perform professional tasks such as disease diagnosis or drug prescription from patient histories. Although the expected benefits are substantial, empirical results show that graph injection based on UMLS usually yields only modest performance gains. This paper systematically reviews existing MQA work, dissecting the selected UMLS sources and their usage with language models, and evaluates each component in isolation. We uncover methodological flaws that have not been discussed before, including low UMLS coverage and weak subgraph retrieval pipelines. Based on these findings, we provide actionable recommendations aimed at unlocking the untapped potential of UMLS as an external knowledge source.
Review