Retrieval Augmented Generation (RAG) is essential for producing accurate, hallucination‑free answers with Large Language Models (LLMs). While LLMs have become better at handling long contexts, they still suffer from the “lost in the middle” problem, making precise retrieval crucial. Existing retrievers split long documents into fixed‑length chunks, discarding the rich global semantic structure of the corpus.
We introduce STAIR, a novel retrieval system that enables an LLM to exploit global structures such as a Table of Contents (ToC) to store and retrieve information efficiently within its parameters. Careful ablation studies on a fine‑tuned Differentiable Search Index (DSI) reveal that incorporating ToC reduces hallucination rates to less than 0.05% and generalizes well even when training samples are scarce.
To foster research on ToC‑based retrieval, we release SearchTome, a diverse benchmark built from 18 books across six domains. STAIR achieves a Recall@1 of 82.6% on SearchTome, outperforming DSI (76.9%) with statistically significant difference. It also surpasses strong baselines such as BM25 (59.5%), DPR (68.7%), and the out‑of‑the‑box Mistral model (13.8%).
Review: STAIR leverages document‑level structures to deliver low‑hallucination, high‑recall retrieval, offering a promising direction for RAG systems. Its robustness in low‑resource settings makes it a valuable foundation for future exploration.