Almost every production retrieval‑augmented question‑answering system hides an interpreter: for each query a language model re‑derives the meaning of the raw corpus and then discards that work. Although per‑token prices have dropped by orders of magnitude, inference spend has risen because context volume grows faster than costs fall. This is the modern equivalent of a full‑table scan, and the remedy is exactly what databases discovered fifty years ago: do the expensive work once at write time, building a maintained structure that makes reads cheap. If a corpus’s read pattern can be known before it ever meets a user, it should be indexed.
We call this paradigm Ingest‑Time Semantic Compilation (ISC): compile a corpus’s meaning into a queryable substrate composed of two coupled layers—incrementally maintained embeddings and atomic claims whose provenance is validated at compile time—and treat that substrate as a first‑class database object with its own DDL, maintenance contract, migration contract, and cost model.
Two existence proofs support ISC. First, substrate upkeep scales with change rather than corpus size: incremental updates are 33.7× cheaper than full reconstruction while preserving floating‑point precision. Second, on a held‑out sample of 500 broadcast‑interview transcripts, compiled claims as the retrieval payload win every one of the 32 budget‑by‑model cells: about 85.2% correct using roughly 2.2k reader tokens versus 72.5% with 16.3k tokens for the best chunk configuration anywhere. The only baseline that keeps pace is a contextualized‑chunk pipeline with hybrid retrieval and reranking, statistically indistinguishable from compiled claims but consuming about twenty‑one times more query‑path tokens – we argue this parity arises because the pipeline itself has begun to compile.
We close by outlining the systems agenda this opens, from compilation planners to read planners, suggesting a new ecosystem built around ingest‑time semantic compilation.
Blogger's Review: ISC pushes the heavy lifting of retrieval systems to the write phase, dramatically cutting query cost and boosting accuracy, and it offers a compelling new architecture for large‑scale RAG deployments.