OA-RAG is an ontology‑based Agentic RAG framework designed to improve the faithfulness of large language model (LLM) answers. During retrieval it incorporates formal reasoning over the ontology, exposing both the logical structure and informal annotations to the LLM. For each generated answer OA-RAG lists the ontology observations used as evidence and reports a model‑assessed grounding level—entailed, supported, related, or unrelated—indicating the strength of that evidence.
The authors evaluated the system on three large‑scale ontologies (e.g., SNOMED CT, MeSH, UMLS) and six real‑world medical QA tasks. Compared with text‑only RAG approaches and ontology‑fine‑tuned baselines, OA‑RAG achieved the highest overall scores, with particularly strong gains on reasoning‑intensive subtasks. Analysis showed that higher grounding levels correlate with higher accuracy, offering a practical cue for answer reliability.
Furthermore, the results demonstrate that ontologies are competitive knowledge sources: on several tasks OA‑RAG outperformed domain‑specific RAG systems that rely on textbooks, confirming the value of ontologies for organizing medical knowledge.
Review