arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.05233cs.RO

MobileManiBench:简化移动操作模型验证

MobileManiBench: Simplifying Model Verification for Mobile Manipulation

Microsoft Research Asia(微软亚洲研究院);University of Sydney(悉尼大学);Tsinghua University(清华大学)

Wenbo Wang, Fangyun Wei, QiXiu Li, Xi Chen, Yaobo Liang, Chang Xu, Jiaolong Yang, Baining Guo

AI总结:

提出仿真优先框架MobileManiBench,基于NVIDIA Isaac Sim和强化学习自动生成多样化移动操作轨迹,包含300K条轨迹,用于验证VLA模型在真实部署前的性能。

摘要:

视觉-语言-动作模型推动了机器人操作的发展,但仍受限于依赖大型、遥操作收集的数据集,这些数据集以静态桌面场景为主。我们提出一个仿真优先框架,用于在实际部署前验证VLA架构,并介绍了MobileManiBench,一个大规模移动机器人操作基准。基于NVIDIA Isaac Sim并由强化学习驱动,我们的流程自动生成多样化的操作轨迹,并带有丰富的标注(语言指令、多视角RGB-深度-分割图像、同步的对象/机器人状态和动作)。MobileManiBench包含2个移动平台(平行夹爪和灵巧手机器人)、2个同步摄像头(头部和右手腕)、20个类别的630个物体、5种技能(打开、关闭、拉、推、拾取),在100个真实场景中执行超过100个任务,生成300K条轨迹。这种设计支持对机器人本体、感知模态和策略架构进行可控、可扩展的研究,加速数据效率和泛化方面的研究。我们对代表性的VLA模型进行了基准测试,并报告了在复杂模拟环境中感知、推理和控制的见解,所有代码、数据集和模型均已公开。

英文摘要:

Vision-language-action models have advanced robotic manipulation but remain constrained by reliance on the large, teleoperation-collected datasets dominated by the static, tabletop scenes. We propose a simulation-first framework to verify VLA architectures before real-world deployment and introduce MobileManiBench, a large-scale benchmark for mobile-based robotic manipulation. Built on NVIDIA Isaac Sim and powered by reinforcement learning, our pipeline autonomously generates diverse manipulation trajectories with rich annotations (language instructions, multi-view RGB-depth-segmentation images, synchronized object/robot states and actions). MobileManiBench features 2 mobile platforms (parallel-gripper and dexterous-hand robots), 2 synchronized cameras (head and right wrist), 630 objects in 20 categories, 5 skills (open, close, pull, push, pick) with over 100 tasks performed in 100 realistic scenes, yielding 300K trajectories. This design enables controlled, scalable studies of robot embodiments, sensing modalities, and policy architectures, accelerating research on data efficiency and generalization. We benchmark representative VLA models and report insights into perception, reasoning, and control in complex simulated environments, with all code, datasets, and models publicly released.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑