We built CRISS (Cancer Registry Intelligent Support System) using a Retrieval‑Augmented Generation (RAG) architecture to give cancer registrars fast, citation‑backed guidance. A domain‑specific knowledge base was created from national cancer registry standards, split into metadata‑tagged passages and indexed as dense embeddings. When a user asks a question, the retrieval component fetches relevant passages, and a large language model (LLM) generates an answer grounded in those citations. For evaluation, we selected easy, medium, and hard registry questions and applied an LLM‑as‑a‑Judge protocol to compare RAG against non‑RAG (pure generation) models across Gemini and GPT families. RAG consistently outperformed non‑RAG, achieving grounding scores of 0.62/0.56/0.59 versus 0.29/0.26/0.29 for easy/medium/hard tiers, and also higher semantic‑similarity scores. Proprietary RAG models were strongest on easy and medium questions, while local open‑source RAG ranked highest on hard questions; proprietary models tended to be more cautious. The study shows that a domain‑specific RAG improves evidence grounding and response quality while preserving human oversight of final coding decisions.
CRISS demonstrates the potential of human‑centered, citation‑grounded AI to support cancer registrars, offering reliable assistance for training and help‑desk scenarios.
Review: By tightly coupling retrieval with generation, the system markedly boosts answer accuracy and traceability in a highly regulated domain, providing a solid technical foundation for compliance and decision support.