VLMs有隧道视野：评估领先VLMs的非局部视觉推理能力
VLMs have Tunnel Vision: Evaluating Nonlocal Visual Reasoning in Leading VLMs
AI总结：
本文评估了领先VLMs的非局部视觉推理能力，发现其在复杂视觉任务上表现不佳，缺乏人类核心视觉推理能力。
摘要：
视觉-语言模型（VLMs）在复杂的视觉任务如视觉问答（VQA）和图表理解方面表现出色，但近期的研究表明它们在简单的感知测试中存在困难。我们提出了对视觉-语言模型非局部视觉推理能力的评估：需要从图像的多个可能遥远区域收集证据并进行链式推理。我们隔离了三种不同的非局部视觉形式：比较感知，需要在工作记忆中保持两幅图像并进行比较；扫视搜索，需要做出离散的、基于证据的跳跃以定位连续的目标；以及平滑视觉搜索，涉及跟随连续的轮廓。旗舰模型（例如GPT-5、Gemini 2.5 Pro、Claude Sonnet 4），即使在先前的原始视觉基准测试中表现良好，也未能通过这些测试，并在我们任务的两个变体上仅略高于随机准确率，而对人类来说这些任务是显而易见的。我们的结构化评估套件使我们能够测试VLMs是否能够执行与人类使用的视觉算法相似的视觉算法。我们的发现表明，尽管在原始视觉敏锐度上有所提升，但当前模型缺乏核心的视觉推理能力。
英文摘要：
Vision-Language Models (VLMs) excel at complex visual tasks such as VQA and chart understanding, yet recent work suggests they struggle with simple perceptual tests. We present an evaluation of vision-language models' capacity for nonlocal visual reasoning: reasoning that requires chaining evidence collected from multiple, possibly distant regions of an image. We isolate three distinct forms of nonlocal vision: comparative perception, which demands holding two images in working memory and comparing them; saccadic search, which requires making discrete, evidence-driven jumps to locate successive targets; and smooth visual search, which involves following a continuous contour. Flagship models (e.g., GPT-5, Gemini 2.5 Pro, Claude Sonnet 4), even those that perform well on prior primitive-vision benchmarks, fail these tests and barely exceed random accuracy on two variants of our tasks that are trivial for humans. Our structured evaluation suite allows us to test whether VLMs can perform visual algorithms similar to those used by humans. Our findings show that despite gains in raw visual acuity, current models lack core visual reasoning capabilities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
