为此生成描述，为彼进行推理：夹在中间的视觉语言模型（VLM）
Caption This, Reason That: VLMs Caught in the Middle
AI总结：
本研究沿感知、注意力、记忆认知轴评估GPT-4o等VLM，发现其在空间理解等任务存在短板，通过视觉-文本解耦分析验证了微调小模型、强化CoT能力的改进效果。
摘要：
近年来，视觉语言模型（VLMs）在视觉理解领域取得了显著进展，但在计数、关系推理等特定视觉任务上仍落后于人类能力。为探究其底层局限性，本研究借鉴认知科学方法论，沿感知、注意力、记忆三大核心认知轴分析VLM性能，使用针对上述能力的任务套件评估包括GPT-4o在内的当前最优VLM。分析揭示了差异化的认知特征：尽管先进模型在类别识别等部分任务上接近性能天花板，但在需要空间理解或选择性注意力的任务中仍存在显著差距。为探究失效原因与潜在改进方法，本研究采用视觉-文本解耦分析，发现直接视觉推理表现不佳的模型，在基于自身生成的文本描述进行推理时性能有明显提升。实验表明，即便在性能持续超越人类的模型中，也亟需提升VLM的思维链（CoT）能力。此外，本研究验证了在复合视觉推理任务上进行针对性微调的潜力，证明对较小规模VLM进行微调可大幅提升核心认知能力。尽管该提升未在高难度分布外基准上转化为显著增强，但本研究证实VLM在本研究数据集上的性能与在其他基准上的性能强相关。本研究详细分析了VLM的认知优劣势，识别了同步感知与推理的关键瓶颈，同时提供了一种有效且简便的解决方案。
英文摘要：
Vision-Language Models (VLMs) have shown remarkable progress in visual understanding in recent years. Yet, they still lag behind human capabilities in specific visual tasks such as counting or relational reasoning. To understand the underlying limitations, we adopt methodologies from cognitive science, analyzing VLM performance along core cognitive axes: Perception, Attention, and Memory. Using a suite of tasks targeting these abilities, we evaluate state-of-the-art VLMs, including GPT-4o. Our analysis reveals distinct cognitive profiles: while advanced models approach ceiling performance on some tasks (e.g. category identification), a significant gap persists, particularly in tasks requiring spatial understanding or selective attention. Investigating the source of these failures and potential methods for improvement, we employ a vision-text decoupling analysis, finding that models struggling with direct visual reasoning show marked improvement when reasoning over their own generated text captions. These experiments reveal a strong need for improved VLM Chain-of-Thought (CoT) abilities, even in models that consistently exceed human performance. Furthermore, we demonstrate the potential of targeted fine-tuning on composite visual reasoning tasks and show that fine-tuning smaller VLMs substantially improves core cognitive abilities. While this improvement does not translate to large enhancements on challenging, out-of-distribution benchmarks, we show broadly that VLM performance on our datasets strongly correlates with performance on these other benchmarks. Our work provides a detailed analysis of VLM cognitive strengths and weaknesses and identifies key bottlenecks in simultaneous perception and reasoning while also providing an effective and simple solution.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
