This work investigates the topology of information‑flow patterns in attention graphs to separate hallucinated from genuine responses. By computing the Forman‑Ricci curvature $\kappa_F$, we expose structural bottlenecks that signal impaired information propagation. Building on this insight, we introduce a technique that captures both semi‑local and global flow characteristics of attention heads, enabling hallucination detection in a single forward pass.
We evaluate the method on several large language models (LLMs) and two established hallucination‑detection benchmarks. Empirically, the proposed single‑pass approach consistently outperforms existing attention‑based multi‑response baselines and remains competitive across diverse model architectures.
A deeper analysis reveals that impaired context sharing among tokens during causal generation is strongly linked to hallucination events. Hallucinated outputs tend to exhibit:
- an over‑reliance on self‑attention;
- diffused retrieval of context from earlier tokens;
- information over‑squashing, especially in the final transformer layer.
These topological signatures pinpoint where the model’s information flow breaks down, offering concrete targets for future mitigation strategies.
Review