视觉-语言模型能识别正方形吗？文本识别在三种模型家族中介导空间推理
Can Vision-Language Models See Squares? Text-Recognition Mediates Spatial Reasoning Across Three Model Families
AI总结：
本研究发现视觉-语言模型在处理非文本视觉元素时存在空间定位能力缺陷，文本识别性能显著优于其本机视觉路径。
摘要：
我们提出了一个简单的实验，揭示了视觉-语言模型（VLMs）的一个基本限制：在缺乏文本标识的情况下，无法准确定位二进制网格中的填充单元格。我们生成了十五个15x15的网格，密度从10.7%到41.8%不等，并将每个网格渲染为两种图像类型——带有文本符号（.和#）的图像和没有网格线的填充正方形。然后要求三种前沿VLMs（Claude Opus、ChatGPT 5.2和Gemini 3 Thinking）进行转录。在文本符号条件下，Claude和ChatGPT的单元格准确率约为91%，F1得分为84%，而Gemini的准确率为84%，F1得分为63%。在填充正方形条件下，三种模型的准确率均降至60-73%，F1得分为29-39%。关键在于所有条件都通过了相同的视觉编码器——文本符号是图像，而不是分词文本。文本与正方形的F1差距在不同模型中从34到54分不等，表明VLMs似乎像拥有一个高保真文本识别路径来处理空间推理，该路径远超其本机视觉路径。每种模型在正方形条件下都表现出不同的失败模式——系统性低估（Claude）、大规模高估（ChatGPT）和模板幻觉（Gemini），但它们都共享相同的底层缺陷：对非文本视觉元素的空间定位严重退化。
英文摘要：
We present a simple experiment that exposes a fundamental limitation in vision-language models (VLMs): the inability to accurately localize filled cells in binary grids when those cells lack textual identity. We generate fifteen 15x15 grids with varying density (10.7%-41.8% filled cells) and render each as two image types -- text symbols (. and #) and filled squares without gridlines -- then ask three frontier VLMs (Claude Opus, ChatGPT 5.2, and Gemini 3 Thinking) to transcribe them. In the text-symbol condition, Claude and ChatGPT achieve approximately 91% cell accuracy and 84% F1, while Gemini achieves 84% accuracy and 63% F1. In the filled-squares condition, all three models collapse to 60-73% accuracy and 29-39% F1. Critically, all conditions pass through the same visual encoder -- the text symbols are images, not tokenized text. The text-vs-squares F1 gap ranges from 34 to 54 points across models, demonstrating that VLMs behave as if they possess a high-fidelity text-recognition pathway for spatial reasoning that dramatically outperforms their native visual pathway. Each model exhibits a distinct failure mode in the squares condition -- systematic under-counting (Claude), massive over-counting (ChatGPT), and template hallucination (Gemini) -- but all share the same underlying deficit: severely degraded spatial localization for non-textual visual elements.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
