看见不等于知道：视觉语言模型是否知道何时不回答空间问题（以及为什么）？
Seeing Isn't Knowing: Do VLMs Know When Not to Answer Spatial Questions (and Why)?
AI总结：
针对视觉语言模型在空间推理中过度自信回答的问题，提出SpatialUncertain框架，通过遮挡和视角歧义两种挑战，评估模型是否知道何时应弃权以及如何寻找可靠证据。
摘要：
空间推理是部署在真实环境中的视觉语言模型（VLM）的基本能力。然而，视觉观察本质上是对3D世界的有限表示：遮挡可能使物体不可见，视角可能使几何属性产生误导。尽管如此，现有的空间推理基准通常假设观察是充分且可靠的，侧重于模型是否产生正确答案，而不是它们是否认识到问题无法回答以及需要哪些额外观察。在这项工作中，我们通过构建一个受控评估框架SpatialUncertain来挑战这一假设，并引入两种观察挑战：（1）遮挡，隐藏目标信息；（2）视角歧义，产生误导性视觉线索。对于每种配置，我们设计在清晰观察下可回答但在引入挑战下需要弃权的空间问题。我们进一步评估模型是否能识别哪些额外视角可以解决视角歧义。我们在多种前沿开源和闭源VLM上的结果揭示了两个一致的失败模式。首先，模型倾向于过度自信地回答，即使在视觉证据不完整或具有误导性时也试图解决空间推理任务，在遮挡下平均准确率约为30%，在视角歧义下低于10%。其次，即使有额外视角可用，一些模型在识别哪些视角能提供可靠证据方面表现接近随机。总之，我们的发现呼吁超越答案正确性，转向评估模型是否知道何时弃权以及如何寻找可靠证据。
英文摘要：
Spatial reasoning benchmarks typically evaluate whether vision-language models can derive the correct answer from a visual observation. Yet in real 3D environments, the observation itself may be unreliable: occlusion can remove task-relevant evidence, while perspective can make visible geometry misleading. Reliable spatial reasoning therefore requires more than answering a question correctly. A model must also assess whether its current observation provides sufficient and trustworthy evidence for that answer. We introduce SPATIALUNCERTAIN, a controlled evaluation framework for studying viewpoint-dependent observational uncertainty. We study two complementary failure modes: missing evidence caused by occlusion and misleading evidence caused by perspective. We further evaluate whether models can recognize when the current view is unreliable and identify a more informative observation. Across eight open- and closed-source vision-language models, we find that model behavior does not track the reliability of visual evidence. Models do not reliably become more cautious as evidence disappears, and under perspective conflict, their judgments increasingly follow projected appearance rather than the unchanged physical 3D relation. Internal analysis suggests a corresponding representational asymmetry: projected 2D relations are readily available, whereas the underlying physical 3D relation is barely decodable. Moreover, models that can identify an informative viewpoint when explicitly asked often fail to recognize when such an additional view is needed. These failures are not fully resolved by prompting or fine-tuning, and providing a better viewpoint is substantially more effective than adding depth information to the same misleading observation. Our results identify assessing the reliability of visual observations as a distinct and missing component of current spatial reasoning evaluation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
