发表机构
HKUST (Guangzhou); The Chinese University of Hong Kong; Knowin AI（香港科技大学（广州）; 香港中文大学; Knowin AI）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究通过具身智能体竞技场评估七个视觉语言模型在机器人任务中的能力，发现Astra在精确估计和接触定位上优势显著，但协调目标导向行动仍是关键差距。
AI 中文摘要
前沿视觉语言模型（VLM）将场景估计、交互基础与可执行动作相结合。理解这些能力如何支持完整的机器人任务，对于评估其作为机器人通才的成熟度至关重要。我们引入了具身智能体竞技场（Embodied Agent Arena），以考察局部能力在几何、空间推理、可供性、任务规划与操作等任务中，是支持还是不足以实现完整的任务成功。该竞技场包含来自32个既有来源的1,000个案例，以及我们的新基准GeoProbe，后者用于在Blender渲染和真实场景图像上进行几何估计。一个极简的测试框架保留了源观测与操作，同时区分了度量精度、功能基础与原生目标完成度。我们评估了七个视觉语言模型，分析了Astra的任务特定优势，并比较了更丰富观测的执行协议与多轮审查。在整个竞技场中，Astra的优势在精确估计与可用接触定位方面最为显著；完成协调的、目标导向的行动仍是实现机器人通才的关键差距。
英文摘要
Frontier vision-language models (VLMs) increasingly estimate scenes, ground interactions, and generate executable actions. How far these native capabilities support embodied generalism across diverse tasks remains unclear. We introduce Embodied Agent Arena to assess seven VLM agents across Geometry, Spatial Reasoning, Affordance, Task Planning, and Manipulation. The arena contains 1,000 cases drawn from 32 established sources and GeoProbe, our new benchmark for geometric estimation on Blender renders and real-scene images. A minimal harness preserves source observations and operations while leaving perception, reasoning, and action selection to the model. Separate measures of metric precision, functional grounding, and native goal completion connect local competence to complete task outcomes. Astra's strengths in precise estimation and usable-contact localization coexist with endpoint errors in tracing and low household-task completion. Supplementary comparisons of richer observations and multi-round review show model- and task-dependent effects. Current VLM agents thus fall short of embodied generalism: they often make partial progress without satisfying all task goals within allotted time and interaction budgets.
Comments44 pages, including appendices. Clarified evaluation scope and expanded related work, benchmark comparisons, and discussion of native capability and task execution; experimental results unchanged. Project page: https://embodied-agent-arena.github.io/embodied-agent-arena/
