当思考有害时：视觉语言模型推理链中的认知信号
When Thinking Hurts: Epistemic Signals in the Reasoning Chains of Visual Language Models
AI总结：
研究视觉语言模型推理链中的认知信号，通过在相同对抗样本上运行四个模型，发现不同模型的答案熵行为模式有差异，思考链熵在部分情况下优于答案熵，还发现结构化弃权及其实用弃权门可提升准确率。
摘要：
视觉语言模型（VLM）的不确定性量化传统上针对答案令牌分布。我们首次对思考模式VLM中的答案熵行为进行了三类别实证表征。在相同的POPE对抗样本上运行四个模型，发现三种定性不同的模式：Qwen3-VL-8B-Thinking显示完全崩溃（答案H AUROC = 0.492）；GLM-4.1V-9B-Thinking未崩溃（0.716）；InternVL3-8B显示选择性思考（仅50%的样本有链，答案H = 0.675全样本/0.602仅思考样本）。在所有三个思考模式模型中，思考链熵在生成链的子集中优于答案熵。在VQAv2问题上，链熵（0.680）也优于答案熵（0.595）。在更难的推理任务中，两个Qwen模型显示适度信号。此外，记录了结构化弃权影响12 - 22%的查询，以及一个实用的弃权门在62.7%的覆盖率下将准确率从71.0%提高到93.8%且无额外推理成本。
英文摘要：
Uncertainty quantification for visual language models (VLMs) conventionally targets the answer token distribution. We provide the first three-family empirical characterisation of answer entropy behaviour in thinking-mode VLMs. Running four models on identical POPE adversarial samples, we find three qualitatively distinct patterns: Qwen3-VL-8B-Thinking shows complete collapse (ans H AUROC = 0.492); GLM-4.1V-9B-Thinking shows no collapse (0.716); and InternVL3-8B shows selective thinking (chains on only 50% of samples, ans H = 0.675 full / 0.602 thinking-only). Across all three thinking-mode models, thinking chain entropy outperforms answer entropy on the subset where chains are generated (0.647, 0.759, 0.608 vs. 0.492, 0.716, 0.602 respectively), suggesting chain signals are the more reliable predictor whenever chains are present. This holds strongly for Qwen and GLM, but with only marginal and statistically unreliable advantage for InternVL3 (n_FP = 17). A 300-sample VQAv2 pilot confirms chain entropy (0.680) outperforms answer entropy (0.595) on VQAv2 questions, with the gap largest for free-form answers (0.733 vs. 0.467). On harder reasoning tasks (HallusionBench) both Qwen models show moderate signal (approx. 0.64), consistent with incomplete pre-commitment on difficult questions. We additionally document structured abstention affecting 12-22% of queries with asymmetry toward absent-object queries, and a practical abstention gate raising accuracy from 71.0% to 93.8% at 62.7% coverage with no additional inference cost.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
