ManipArena: 通用机器人操作的综合现实世界评估
ManipArena: Comprehensive Real-world Evaluation of Reasoning-Oriented Generalist Robot Manipulation
AI总结：
ManipArena通过标准化框架解决现有评估不足问题，提供真实场景下的多任务测试与多级泛化能力，支持长时移动操作与传感诊断，促进视觉-语言-动作模型和世界模型的发展。
摘要：
视觉-语言-动作（VLA）模型和世界模型最近成为通用机器人智能的有希望范式，但其进展受缺乏可靠评估协议的阻碍，这些协议无法反映真实部署。现有基准主要基于模拟器，提供可控性但无法捕捉感知噪声、复杂接触动态、硬件限制和系统延迟导致的真实差距。此外，不同机器人平台的碎片化现实世界评估阻碍了公平和可重复的比较。为解决这些挑战，我们引入ManipArena，一种标准化评估框架，旨在弥合模拟与现实执行之间的鸿沟。ManipArena包含20种多样任务，涵盖10,812条专家轨迹，强调需要语义和空间推理的推理导向操作任务，支持通过受控的分布外设置进行多级泛化，并包含长时移动操作，超越桌面场景。该框架进一步提供丰富的感官诊断，包括低级电机信号，并通过高质量3D扫描构建同步现实到模拟环境。这些特性共同使VLA和世界模型方法能够公平、真实和可重复地评估，为诊断和推进具身智能系统提供可扩展的基础。
英文摘要：
Vision-Language-Action (VLA) models and world-action models have emerged as central paradigms for general-purpose robotic intelligence, yet their empirical progress remains constrained by the absence of evaluation protocols that are both physically realistic and diagnostically controlled. Simulator-centric benchmarks provide scale and reproducibility, but cannot fully capture the reality gap induced by perception noise, contact dynamics, latency, calibration error, and hardware constraints. Conversely, real-robot evaluations are often fragmented across platforms, scenes, objects, and scoring rules, making fair comparison and failure attribution difficult. We introduce ManipArena, a standardized real-robot evaluation framework for studying manipulation generalization under matched physical conditions. ManipArena comprises 20 tasks, 10,812 expert trajectories, 13.5M frames, and approximately 188 robot hours across tabletop and mobile manipulation. The framework combines schema-defined task variation, stratified in-domain, visualshift, and semantic-OOD trials, subtask-level partial-credit scoring, three-level language annotations, low-level motor signals, and paired real-to-sim environments reconstructed from physical scenes. Using ManipArena, we evaluate seven tabletop configurations spanning VLA and world-action-model policies. The results show that real-robot conclusions depend not only on architecture, but also on model provenance, fine-tuning regime, data sampling, and annotation granularity. ManipArena thus provides a reproducible and interpretable foundation for diagnosing capability boundaries and failure modes in embodied generalization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
