arXiv 2510.178012026-07-07cs.ROcs.CV

Robobench：作为具身大脑的多模态大语言模型的综合评估基准

Robobench: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models as Embodied Brain

State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（信息多媒体国家重点实验室，计算机学院，北京大学）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；Institute for Brain and Intelligence, Fudan University（脑与智能研究院，复旦大学）；University of Science and Technology Beijing（北京科技大学）；Beijing Innovation Center of Humanoid Robotics（北京人形机器人创新中心）

Yulin Luo, Chun-Kai Fan, Menghang Dong, Jiayu Shi, Xiangju Mi, Mengdi Zhao, Bo-Wen Zhang, Cheng Chi, Jiaming Liu, Gaole Dai, Rongyu Zhang, Ruichuan An, Kun Wu, … 展开作者

Yulin Luo, Chun-Kai Fan, Menghang Dong, Jiayu Shi, Xiangju Mi, Mengdi Zhao, Bo-Wen Zhang, Cheng Chi, Jiaming Liu, Gaole Dai, Rongyu Zhang, Ruichuan An, Kun Wu, Zhengping Che, Shaoxuan Xie, Guocai Yao, Zhongxia Zhao, Pengwei Wang, Guang Liu, Zhongyuan Wang, Tiejun Huang, Shanghang Zhang

备注：: ECCV 2026 Camera Ready

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

研究针对在动态非结构化环境中构建能感知、推理和行动的机器人的挑战，介绍RoboBench基准，涵盖多维度、多任务等，能评估多模态大语言模型，揭示其局限性并分析与下游控制关系，为量化高级认知提供支架。

摘要：

构建能在动态、非结构化环境中感知、推理和行动的机器人仍是核心挑战。近期具身系统常采用双系统范式，我们将系统2视为具身大脑。现有基准存在局限，我们引入RoboBench评估多模态大语言模型。它涵盖五个维度，有多种任务和问答对，源于真实机器人数据等。实验揭示模型局限，还分析了具身认知能力与下游控制的关系。

英文摘要：

Building robots that can perceive, reason, and act in dynamic, unstructured environments remains a central challenge. Recent embodied systems often follow a dual-system paradigm, where System 2 performs high-level reasoning and System 1 handles low-level control. We refer to System 2 as the embodied brain, the cognitive core for decision-making in manipulation. Although evaluating this embodied brain is crucial, existing benchmarks mainly measure execution success or cover only limited aspects of high-level cognition and task realism. We introduce RoboBench, a benchmark for evaluating multimodal large language models (MLLMs) as embodied brains. RoboBench covers five dimensions: Instruction Comprehension, Perception Reasoning, Generalized Planning, Affordance Prediction, and Failure Analysis. It spans 14 capabilities, 25 tasks, and 6,092 QA pairs. To improve realism, it draws from large-scale real robotic data and in-house collection across diverse embodiments, attribute-rich objects, multi-view scenes, and memory-driven navigation. For planning, RoboBench introduces an MLLM-as-world-simulator framework that assesses whether predicted plans can achieve critical object-state changes under physical and visual constraints, enabling more faithful evaluation of long-horizon reasoning than symbolic matching. Experiments on 18 state-of-the-art MLLMs reveal persistent limitations in implicit instruction understanding, spatiotemporal reasoning, cross-scenario planning, fine-grained affordance understanding, and failure diagnosis. We further analyze how embodied cognitive abilities relate to downstream robotic control. RoboBench offers a comprehensive scaffold for quantifying high-level cognition and guiding next-generation MLLMs toward more robust robotic intelligence.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。