超越定位：从全景图像中对MLLMs的视角条件空间推理进行综合诊断
Beyond Localization: A Comprehensive Benchmark of Perspective-Conditioned Spatial Reasoning in MLLMs from Omnidirectional Images
AI总结：
本文研究了多模态大语言模型（MLLMs）在视角变化下的空间推理能力，提出了PCSR-Bench基准测试，评估了14种代表性MLLMs在不同任务上的表现，揭示了空间推理能力的显著差距，并探讨了通过强化学习进行优化的可能性。
摘要：
多模态大语言模型（MLLMs）在视觉感知方面表现出色，但在视角变化下的空间推理能力有限。本文将这一挑战定义为视角条件空间推理（PCSR），研究了在360度全景图像中，广阔的场景覆盖减少了部分观察的歧义，但并不消除视角依赖推理的必要性。为了评估这一能力，我们引入了PCSR-Bench，这是一个包含来自2600张全景图像、26种室内环境的84,373个问题-答案对的诊断基准。PCSR-Bench包含八个任务，涵盖基础感知（如物体计数、相对距离和相对方向）和高级PCSR，包括组合链、以自身为中心的旋转、视角重新锚定、自身扭曲和有限视角可见性。我们评估了14种代表性MLLMs，并观察到显著的感知-推理差距：在基础相对方向任务上准确率达到57.59%，但在以自身为中心的旋转任务上降至13.49%，在以自身为中心的扭曲任务上降至7.13%，在开放性组合推理任务上降至0.64%。为了探索这一差距的可塑性，我们对一个7B规模的模型进行了基于强化学习的诊断研究。奖励塑造将匹配的7B基线从31.10%提升到60.06%，表明PCSR是部分可塑性而非完全不可变。然而，这些收益是任务选择性的，对奖励设计敏感，包括权重分配和奖励制定，并在一定程度上依赖于评估协议。这些结果将PCSR定位为当前MLLMs的关键瓶颈，并突显了在有针对性优化下的有限但有意义的恢复空间。
英文摘要：
Multimodal Large Language Models (MLLMs) show strong visual perception, yet remain limited in reasoning about space under changing viewpoints. We study this challenge as Perspective-Conditioned Spatial Reasoning (PCSR) in 360 degree omnidirectional images, where broad scene coverage reduces ambiguity from partial observations without eliminating the need for viewpoint-dependent inference. To assess this capability, we introduce PCSR-Bench, a diagnostic benchmark of 84,373 QA pairs from 2,600 omnidirectional images across 26 indoor environments, organized into eight tasks under three cognitive groups--- Perception, Spatial, and advanced PCSR. We evaluate 14 representative MLLMs and observe a substantial perception--reasoning gap: accuracy reaches 57.59% on Limited Field-of-View Reasoning (T7) but drops to 13.49%, 7.13%, and 0.64% on Relative Direction (T2), Egocentric Rotation (T4), and open-ended Compositional Directional Chains (T3), respectively. To probe the plasticity of this gap, we conduct an RL-based diagnostic study on a 7B-scale model. Reward shaping improves a matched 7B baseline from 31.10% to 60.06% under a controlled setting, suggesting that PCSR exhibits partial plasticity rather than being fully immutable. Still, these gains are task-selective, sensitive to reward design, and partially dependent on the evaluation protocol. These results position PCSR as a key bottleneck in current MLLMs and highlight meaningful yet bounded room for recovery under targeted optimization. Details and access are available at https://github.com/Caleb-ychen/PCSR-Benchmark.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
