Standardizing clinical language to SNOMED CT improves interoperability, analytics, and reusable phenotyping, yet recommending concepts remains difficult when relevant concepts are rare or absent from training data. We built a masked‑concept recommendation benchmark using the SNOMED CT Entity Linking Challenge v1.2.1 derived from MIMIC‑IV‑Note. The dataset contains 75,491 annotations across 272 discharge summaries, with 204 notes for training and 68 for historical testing. For each unique note‑concept pair, the target mention is masked in a local clinical context, and the system must rank SNOMED CT concepts observed during training.
We compared a popularity baseline, sparse TF‑IDF concept prototypes, dense latent semantic analysis (LSA) embeddings, sparse‑dense fusion, retrieved‑note evidence, and a retrieval‑augmented hybrid. Sparse TF‑IDF performed best, achieving Recall@1 of 14.81%, Recall@10 of 33.43%, MRR of 0.2114, and nDCG@10 of 0.2297. Retrieval augmentation did not improve this baseline, with Recall@10 of 31.99% and MRR of 0.1937. Concept frequency strongly affected performance: Recall@10 was 7.74% for concepts appearing in only one or two training notes versus 43.90% for concepts appearing in more than ten notes. Moreover, 9.66% of test note‑concept pairs contained concepts unseen during training. These findings show that in low‑resource settings, local lexical context and terminology coverage are major determinants of recommendation quality, providing a reproducible baseline for future ontology‑grounded and biomedical‑encoder retrieval systems.
Review