空间中的注意：VLM头部在空间推理中的功能角色
Attention in Space: Functional Roles of VLM Heads for Spatial Reasoning
AI总结：
本文研究VLM中注意力头部在空间推理中的作用，通过机制可解释性视角分析其功能，提出CogVSR数据集并开发探针框架，揭示注意力头部在空间推理中的稀疏性和关键作用。
摘要：
尽管大型视觉-语言模型（VLMs）在视觉和语言任务上取得了显著进展，但空间推理仍是一个持续的挑战。本文研究了VLM中注意力头部如何通过机械可解释性视角分析其在空间推理中的功能作用。我们引入了CogVSR数据集，该数据集将复杂的空间推理问题分解为逐步的子问题，以模拟人类样式的推理过程，每个子问题都与特定的认知功能如空间感知或关系推理相关联。基于CogVSR，我们开发了一种探针框架，以识别和表征这些功能性的注意力头部。我们的分析表明，这些功能性的注意力头部在不同的VLM家族中是稀疏的，数量和分布因功能而异。值得注意的是，专门用于空间推理的头部数量少于其他认知功能的头部，突显了其稀缺性。我们提出方法以激活潜在的空间头部，提高空间理解能力。干预实验进一步证明了它们在空间推理中的关键作用：移除功能性头部会导致性能下降，而强调它们则会提高准确性。本研究提供了关于VLM如何关注空间的新可解释性洞察，并为增强多模态模型中的复杂空间推理铺平了道路。
英文摘要：
Despite remarkable advances in large Vision-Language Models (VLMs), spatial reasoning remains a persistent challenge. In this work, we investigate how attention heads within VLMs contribute to spatial reasoning by analyzing their functional roles through a mechanistic interpretability lens. We introduce CogVSR, a dataset that decomposes complex spatial reasoning questions into step-by-step subquestions designed to simulate human-like reasoning via a chain-of-thought paradigm, with each subquestion linked to specific cognitive functions such as spatial perception or relational reasoning. Building on CogVSR, we develop a probing framework to identify and characterize attention heads specialized for these functions. Our analysis across diverse VLM families reveals that these functional heads are universally sparse, vary in number and distribution across functions. Notably, spatially specialized heads are fewer than those for other cognitive functions, highlighting their scarcity. We propose methods to activate latent spatial heads, improving spatial understanding. Intervention experiments further demonstrate their critical role in spatial reasoning: removing functional heads leads to performance degradation, while emphasizing them enhances accuracy. This study provides new interpretability driven insights into how VLMs attend to space and paves the way for enhancing complex spatial reasoning in multimodal models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
