In visual language models (VLMs), conventional uncertainty quantification primarily targets the answer token distribution. This paper provides the first empirical characterization of answer entropy behavior across three models in thinking mode.
Running four models on identical POPE adversarial samples reveals three qualitatively distinct patterns: Qwen3-VL-8B-Thinking shows complete collapse (ans H AUROC = 0.492); GLM-4.1V-9B-Thinking shows no collapse (0.716); and InternVL3-8B exhibits selective thinking (chains generated on only 50% of samples, ans H = 0.675 full / 0.602 thinking-only).
Across all three thinking-mode models, thinking chain entropy outperforms answer entropy on the subset where chains are generated (0.647, 0.759, 0.608 vs. 0.492, 0.716, 0.602), suggesting that chain signals are more reliable predictors when chains are present. This is particularly strong for Qwen and GLM, but with only marginal and statistically unreliable advantage for InternVL3 (n_FP = 17).
A 300-sample VQAv2 pilot confirms that chain entropy (0.680) outperforms answer entropy (0.595) on VQAv2 questions, with the largest gap for free-form answers (0.733 vs. 0.467).
On harder reasoning tasks (HallusionBench), both Qwen models show moderate signal (approx. 0.64), consistent with incomplete pre-commitment on difficult questions. Additionally, we document structured abstention affecting 12-22% of queries with asymmetry toward absent-object queries, and a practical abstention gate raises accuracy from 71.0% to 93.8% at 62.7% coverage with no additional inference cost.
Blogger's Review: This paper provides a profound analysis of different visual language models under thinking mode, revealing the significance of reasoning chain entropy in predictive tasks, especially in complex problem-solving contexts. It opens new avenues for model optimization and uncertainty assessment methodologies in future research.