不要眨眼：多模态推理中的证据崩溃
Don't Blink: Evidence Collapse during Multimodal Reasoning
AI总结：
研究发现多模态推理过程中证据逐渐消失的现象，提出通过视觉-熵交互模型降低风险，提升模型在分布偏移下的安全性。
摘要：
推理视觉语言模型在逐步失去视觉基础的同时变得更加准确，这导致了任务条件下的危险区域，其中低熵预测显得自信但无根据，文本-only 监控无法检测此类失败模式。在MathVista、HallusionBench和MMMU_Pro上评估三个推理VLMs，发现普遍的证据崩溃现象：随着推理展开，对注释证据区域的注意力大幅下降，往往失去超过一半的证据质量。全响应熵是跨数据集转移下最可靠的文本-only 不确定性信号，但添加视觉特征时使用单一全局线性规则则脆弱且常导致转移退化。一个熵-视觉交互模型揭示了任务条件下的模式：低熵且视觉不参与的预测在持续视觉参考任务中具有危险性，但在符号任务中则无害。利用此结构，一种针对性视觉 veto 可将选择性风险降低至90%覆盖率下的1.9个百分点，同时避免预期中的退化。结果支持了任务感知的多模态监控，以在分布偏移下安全部署。
英文摘要：
Reasoning VLMs can become more accurate while progressively losing visual grounding as they think. This creates task-conditional danger zones where low-entropy predictions are confident but ungrounded, a failure mode text-only monitoring cannot detect. Evaluating three reasoning VLMs on MathVista, HallusionBench, and MMMU_Pro, we find a pervasive evidence-collapse phenomenon: attention to annotated evidence regions drops substantially, often losing over half of evidence mass, as reasoning unfolds. Full-response entropy is the most reliable text-only uncertainty signal under cross-dataset transfer, yet adding vision features with a single global linear rule is brittle and often degrades transfer. An entropy-vision interaction model reveals a task-conditional regime: lowentropy, visually disengaged predictions are hazardous on sustained visual-reference tasks but benign on symbolic tasks. Using this structure, a targeted vision veto reduces selective risk by up to 1.9 percentage points at 90% coverage, while avoiding degradations where disengagement is expected. The results support task-aware multimodal monitoring for safe deployment under distribution shift.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
