Abstract
Human visual reasoning typically follows a coarse-to-fine attention process, starting from global scene understanding and gradually focusing on question-relevant regions. However, multimodal large language models may deviate from this pattern due to attention drift and the underutilization of visual evidence, which can lead to hallucinations. To mitigate these issues, this study proposes a Dual-Indicator Guided Contrastive Alignment (DICA), which tracks two information-theoretic indicators during inference: Visual Attention Entropy (VAE), reflecting the concentration of visual attention, and Output Image Correlation (OIC), measuring the dependence of generated outputs on the visual input.
An abnormal increase in VAE or a decrease in OIC corresponds to different failure modes, triggering targeted contrastive alignment to restore visual grounding. Experimental results across multiple benchmarks demonstrate that DICA consistently outperforms existing approaches and substantially reduces hallucinations, highlighting the effectiveness of indicator-driven intervention in improving multimodal inference reliability. The code is publicly available at GitHub.
Blogger's Review: The introduction of DICA effectively addresses the visual reasoning issues in multimodal large language models. By incorporating information-theoretic indicators, it strengthens the visual grounding of the model, making inference more reliable. The method's practicality and effectiveness have been validated across multiple benchmarks, making it a worthy subject for further research and application.