arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.06678cs.ROcs.AIcs.CVcs.LG

M3Bench:在3D场景中面向移动操作的全身运动生成基准测试

M3Bench: Benchmarking Whole-body Motion Generation for Mobile Manipulation in 3D Scenes

Huazhong University of Science and Technology(华中科技大学);Peking University(北京大学)

Zeyu Zhang, Sixu Yan, Muzhi Han, Zaijin Wang, Xinggang Wang, Song-Chun Zhu, Hangxin Liu

AI总结:

本文提出M3Bench基准及其数据生成工具M3BenchMaker,用于3D场景中移动操作的全身运动生成,包含3万个任务,评估显示现有先进模型在协调全身运动和遵守约束方面仍存在困难。

摘要:

我们提出了M3Bench,一个用于移动操作任务中全身运动生成的新基准。给定3D场景上下文,M3Bench要求具身智能体推理其自身配置、环境约束和任务目标,以生成用于物体重排的协调全身运动轨迹。M3Bench包含跨越119个不同场景的30,000个物体重排任务,并提供由我们新开发的M3BenchMaker生成的专家演示,这是一种自动数据生成工具,仅使用基本场景和机器人信息即可从高级任务指令生成全身运动轨迹。我们的基准包含各种任务划分以评估跨不同维度的泛化能力,并利用逼真的物理模拟进行轨迹评估。广泛的评估分析表明,最先进的模型在协调底盘-手臂运动的同时难以遵守环境和任务特定约束,凸显了需要新模型来弥合这一差距。通过发布M3Bench和M3BenchMaker,我们旨在推动机器人研究向更多样化、真实世界环境中更具适应性和能力的移动操作发展。

英文摘要:

We propose M3Bench, a new benchmark for whole-body motion generation in mobile manipulation tasks. Given a 3D scene context, M3Bench requires an embodied agent to reason about its configuration, environmental constraints, and task objectives to generate coordinated whole-body motion trajectories for object rearrangement. M3Bench features 30,000 object rearrangement tasks across 119 diverse scenes, providing expert demonstrations generated by our newly developed M3BenchMaker, an automatic data generation tool that produces whole-body motion trajectories from high-level task instructions using only basic scene and robot information. Our benchmark includes various task splits to evaluate generalization across different dimensions and leverages realistic physics simulation for trajectory assessment. Extensive evaluation analysis reveals that state-of-the-art models struggle with coordinating base-arm motion while adhering to environmental and task-specific constraints, underscoring the need for new models to bridge this gap. By releasing M3Bench and M3BenchMaker we aim to advance robotics research toward more adaptive and capable mobile manipulation in diverse, real-world environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑