Engineering standards are hierarchical, multimodal, and densely linked by figures, tables, cross‑references, and normative rules, which makes them difficult knowledge sources for large language models. Existing retrieval‑augmented generation (RAG) approaches mainly rely on unstructured text retrieval and struggle to jointly reason over textual provisions, visual specifications, and document structure. GEMS‑RAG addresses this gap by providing a graph‑grounded, multimodal reasoning framework. The system consists of hierarchical parsing that breaks a standard into sections and clauses; fine‑grained visual entity extraction that decomposes figures into symbols, dimension annotations and other entities; multimodal knowledge‑graph construction where nodes represent textual provisions, visual entities, engineering concepts and regulatory rules, and edges capture references and containment; and graph‑grounded hybrid retrieval that combines dense vector search, sparse inverted‑index search, and visual entity search, followed by graph expansion to gather cross‑modal evidence. Evaluation on the Manual on Uniform Traffic Control Devices (MUTCD)—which contains regulations, technical drawings, and standardized traffic‑control devices—shows that GEMS‑RAG outperforms text‑only, multimodal, and pure‑graph RAG baselines in both retrieval effectiveness and question‑answering accuracy. The framework is applicable to other engineering codes, technical manuals, regulatory documents, and safety‑critical scientific standards.
Review