Adoption of large language models (LLMs) in enterprise finance, especially in regulated workflows such as Financial Planning and Analysis (FP&A), is limited more by trust than by fluency. An answer is only usable when it can be traced back to authoritative sources and audited after the fact.
This paper introduces the Knowledge‑Driven Analytics Framework (KDAF) and argues that retrieval‑augmented generation (RAG) for enterprise finance should be evaluated on auditability as well as accuracy. KDAF builds ontology‑driven knowledge bases through six iterative stages and retrieves evidence via Context‑Aware Relevance Propagation (CARP), ensuring that every retrieved fact carries its relationship type, confidence score, and source lineage.
The authors evaluate the framework on FinanceBench (145 finance‑related questions), comparing KDAF against zero‑context inference, BM25, concept‑weighted lexical retrieval, and ungrounded graph traversal. Key findings include:
- Retrieval is essential: zero‑context inference achieves only 4.1% correctness, whereas retrieval‑augmented settings reach 10‑12%.
- On answer correctness, KDAF and BM25 are statistically indistinguishable (difference -0.007, 95% CI [-0.021, 0.000]), indicating that accuracy alone does not justify the added complexity of structured retrieval.
- On auditability, KDAF leads with a citation traceability F1 of 0.515, surpassing ungrounded traversal by +0.027 (CI [0.006, 0.050]) and BM25 by +0.052 (CI [0.024, 0.083]), with confidence intervals excluding zero.
- Graph‑structured retrieval completely avoids evidence unrelated to the question’s subject entity (0/426 items), whereas lexical baselines include unrelated evidence at rates of 16.8% and 20.2%. Every selected item resolves to a full provenance chain.
The authors conclude that auditability, not raw accuracy, is the primary axis on which ontology‑grounded retrieval earns its cost. By coupling a rigorously defined ontology with CARP, KDAF delivers auditable answers, offering a trustworthy pathway for LLM deployment in enterprise finance.
Blogger's Review: The paper convincingly ties regulatory compliance to technical design, presenting a clear, repeatable methodology for building auditable LLM pipelines. The quantitative focus on traceability metrics fills a notable gap in current LLM evaluation practices. Future work could explore cross‑domain ontology transfer and real‑time provenance visualization to further enhance practical applicability.