VLMs 是在看还是只是在说？揭示视觉重新检查的幻觉
Are VLMs Seeing or Just Saying? Uncovering the Illusion of Visual Re-examination
AI总结：
通过图像交换探测框架 VisualSwap 和 800 对图像基准 VS-Bench，发现视觉语言模型在推理时声称的“重新检查图像”多为文本模式，而非真正的视觉重新检查，且思考模型更易受影响，用户指令可恢复视觉基础但自我反思无效。
摘要：
视觉语言模型（VLM）在推理过程中经常产生自我反思的语句，如“让我再检查一下图片”。这样的语句是否触发了真正的视觉重新检查，还是仅仅是习得的文本模式？我们通过 VisualSwap（一种图像交换探测框架）对此进行研究：在模型对一张图像进行推理后，我们将其替换为视觉上相似但语义不同的图像，并测试模型是否注意到这一变化。我们引入了 VS-Bench，包含从 MathVista、MathVerse、MathVision 和 MMMU-Pro 中精选的 800 对图像。在 Qwen3-VL、Kimi-VL 和 ERNIE-VL 上的实验揭示了一个惊人的失败：模型绝大多数情况下忽略了图像交换，准确率下降高达 60%。与直觉相反，思考模型比其指令对应模型脆弱近 3 倍，且扩展规模无法缓解。多轮用户指令可以恢复视觉基础，但连续生成过程中自我生成的反思语句则不能。注意力分析解释了原因：用户指令显著提高了对视觉标记的注意力，而自我反思则没有。当前的 VLM 在声称执行视觉重新检查时倾向于“说”而非真正“看”。我们的代码和数据集可在项目页面获取：https://visualswap.github.io
英文摘要：
Vision-Language Models (VLMs) often produce self-reflective statements like "let me check the figure again" during reasoning. Do such statements trigger genuine visual re-examination, or are they merely learned textual patterns? We investigate this via VisualSwap, an image-swap probing framework: after a model reasons over an image, we replace it with a visually similar but semantically different one and test whether the model notices. We introduce VS-Bench, 800 image pairs curated from MathVista, MathVerse, MathVision, and MMMU-Pro. Experiments on Qwen3-VL, Kimi-VL, and ERNIE-VL reveal a striking failure: models overwhelmingly miss the swap, with accuracy dropping by up to 60%. Counterintuitively, thinking models are nearly 3x more vulnerable than their instructed counterparts, and scaling offers no mitigation. Multi-turn user instructions restore visual grounding, but self-generated reflective statements during continuous generation do not. Attention analysis explains why: user instructions substantially elevate attention to visual tokens, whereas self-reflection does not. Current VLMs tend to say rather than actually see when claiming to perform visual re-examination. Our code and dataset are available at the project page: https://visualswap.github.io
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
