跨视角视觉：评估视觉-语言模型在机器人场景中的空间推理能力
Seeing Across Views: Benchmarking Spatial Reasoning of Vision-Language Models in Robotic Scenes
AI总结：
本文提出MV-RoboBench基准，评估视觉-语言模型在机器人场景中的多视角空间推理能力，揭示其在多视角机器人感知中的挑战。
摘要：
视觉-语言模型（VLMs）是具身人工智能的关键组成部分，使机器人能够在复杂环境中感知、推理和行动。它们也是最近的视觉-语言-动作（VLA）模型的基础。然而，大多数对VLMs的评估都集中在单视角设置上，使其整合多视角信息的能力未被充分探索。同时，多摄像头设置在机器人平台中越来越标准，因为它们提供了互补的视角以缓解遮挡和深度模糊的问题。因此，VLMs是否能有效利用这些多视角输入进行机器人推理仍然是一个开放性问题。为了填补这一空白，我们引入了MV-RoboBench，这是一个专门设计用于评估VLMs在机器人操作中的多视角空间推理能力的基准。MV-RoboBench包含1700多个手工整理的QA问题，涵盖八个子任务，分为两个主要类别：空间理解和机器人执行。我们评估了一组多样化的现有VLMs，包括开源和闭源模型，以及结合了CoT启发技术的增强版本。结果表明，最先进的模型仍然远低于人类表现，凸显了VLMs在多视角机器人感知中面临的重大挑战。此外，我们的分析揭示了两个关键发现：（i）在多视角机器人场景中，空间智能和机器人任务执行呈正相关；（ii）在现有通用单视角空间理解基准上表现优异的模型在我们基准评估的机器人空间任务中并不一定成功。我们发布MV-RoboBench作为开放资源，以促进空间基础的VLMs和VLAs的进步，不仅提供数据，还提供标准化的多视角具身推理评估协议。
英文摘要：
Vision-language models (VLMs) are essential to Embodied AI, enabling robots to perceive, reason, and act in complex environments. They also serve as the foundation for the recent Vision-Language-Action (VLA) models. Yet most evaluations of VLMs focus on single-view settings, leaving their ability to integrate multi-view information underexplored. At the same time, multi-camera setups are increasingly standard in robotic platforms, as they provide complementary perspectives to mitigate occlusion and depth ambiguity. Whether VLMs can effectively leverage such multi-view inputs for robotic reasoning therefore remains an open question. To bridge this gap, we introduce MV-RoboBench, a benchmark specifically designed to evaluate the multi-view spatial reasoning capabilities of VLMs in robotic manipulation. MV-RoboBench consists of 1.7k manually curated QA items across eight subtasks, divided into two primary categories: spatial understanding and robotic execution. We evaluate a diverse set of existing VLMs, including both open-source and closed-source models, along with enhanced versions incorporating CoT-inspired techniques. The results show that state-of-the-art models remain far below human performance, underscoring the substantial challenges VLMs face in multi-view robotic perception. Additionally, our analysis uncovers two key findings: (i) spatial intelligence and robotic task execution are positively correlated in multi-view robotic scenarios; and (ii) strong performance on existing general-purpose single-view spatial understanding benchmarks does not reliably translate to success in the robotic spatial tasks assessed by our benchmark. We release MV-RoboBench as an open resource to foster progress in spatially grounded VLMs and VLAs, providing not only data but also a standardized evaluation protocol for multi-view embodied reasoning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
