Visual interventions offer a training‑free way to curb hallucinations in multimodal large language models (LLMs). We observe that applying the same visual‑residual edit at successive decoder layers yields markedly different leverage: the effect is strong in early layers but drops sharply in deeper ones. To quantify this, we matched the edit’s RMS strength across depth and applied it layer‑by‑layer on six checkpoints from three model families. A consistent decline in effect defines an Evidence Anchoring Boundary (EAB). Fixing the boundary on calibration queries, we evaluated disjoint test queries and found that effects before the EAB are $13.78\times$–$26.10\times$ larger than those after it. Comparisons with intermediate readouts show that the edit can lose leverage before answer preferences stabilize.
We also compare counterfactual subtraction across presence, attribute, and relation questions. Subtraction may correct a misleading preference, leave the decision unchanged, or erase a preference that supports the correct answer. Building on these findings, we introduce calibrated selective routing: the original prediction is kept unless calibration indicates that an intervention is beneficial for the question type. This improves full‑set attribute accuracy by $1.56$–$1.81$ percentage points across three checkpoints, whereas universal subtraction is neutral or harmful.
Review: The paper systematically pinpoints the depth at which visual interventions remain effective and proposes a calibration‑driven selective routing mechanism, demonstrating that targeted visual cues can enhance multimodal LLM reliability without degrading existing performance.