ESPIRE：一种用于视觉-语言模型具身空间推理的诊断基准
ESPIRE: A Diagnostic Benchmark for Embodied Spatial Reasoning of Vision-Language Models
AI总结：
本文提出ESPIRE基准，通过模拟环境评估视觉-语言模型在具身空间推理中的表现，改进了传统评估方法，实现了更细致的推理与行动分析。
摘要：
近年来，视觉-语言模型（VLMs）在增强其空间认知以适应具身领域方面取得了进展。尽管有所进步，现有评估在范式和覆盖范围上都受到限制，阻碍了快速迭代模型的发展。为了解决这些限制，我们提出了ESPIRE，一种用于具身空间推理的诊断基准。ESPIRE提供了一个物理上接地的模拟世界，使VLMs能够在此环境中进行空间推理相关的机器人任务评估，从而缩小评估与现实世界部署之间的差距。为了使VLMs适应机器人任务，我们将每个任务分解为定位和执行，并将两者均作为生成性问题来处理，这与主流的判别性评估（如通过视觉问答）形成鲜明对比，后者依赖于干扰项并丢弃执行。这种分解进一步使我们能够超越被动的空间推理，向推理与行动分析迈进。我们系统地设计了ESPIRE，既在指令层面又在环境层面，确保了空间推理场景的广泛覆盖。我们使用ESPIRE诊断了一系列前沿的VLMs，并对其空间推理行为进行了深入分析。
英文摘要：
A recent trend in vision-language models (VLMs) has been to enhance their spatial cognition for embodied domains. Despite progress, existing evaluations have been limited both in paradigm and in coverage, hindering rapid, iterative model development. To address these limitations, we propose ESPIRE, a diagnostic benchmark for embodied spatial reasoning. ESPIRE offers a simulated world that physically grounds VLMs and evaluates them on spatial-reasoning-centric robotic tasks, thus narrowing the gap between evaluation and real-world deployment. To adapt VLMs to robotic tasks, we decompose each task into localization and execution, and frame both as generative problems, in stark contrast to predominant discriminative evaluations (e.g., via visual-question answering) that rely on distractors and discard execution. This decomposition further enables a fine-grained analysis beyond passive spatial reasoning toward reasoning to act. We systematically design ESPIRE both at the instruction level and at the environment level, ensuring broad coverage of spatial reasoning scenarios. We use ESPIRE to diagnose a range of frontier VLMs and provide in-depth analysis of their spatial reasoning behaviors.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
