arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.10028cs.CVcs.RO
收藏

3D动态感知操控:赋予操控策略三维前瞻性

3D Dynamics-Aware Manipulation: Endowing Manipulation Policies with 3D Foresight

The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));JAKA Robotics Co., Ltd.(JAKA机器人有限公司)

Yuxin He, Ruihao Zhang, Xianzu Wu, Zhiyuan Zhang, Cheng Ding, Qiang Nie

AI总结:

本文提出一种融合3D世界建模与策略学习的框架,通过引入三个自监督任务提升操控策略的3D前瞻性,实验表明在仿真和现实环境中显著提升操控性能。

摘要:

将世界建模纳入操控策略学习中推动了操控性能的边界。然而,现有方法仅建模2D视觉动态,无法应对涉及显著深度运动的目标任务。为此,我们提出一种3D动态感知操控框架,无缝整合3D世界建模与策略学习。引入三个自监督学习任务(当前深度估计、未来RGB-D预测、3D流预测),这些任务相互补充,赋予策略模型3D前瞻性。在仿真和现实世界中的广泛实验表明,3D前瞻性能显著提升操控策略性能,而无需牺牲推理速度。代码可在https://github.com/Stardust-hyx/3D-Foresight获取。

英文摘要:

The incorporation of world modeling into manipulation policy learning has pushed the boundary of manipulation performance. However, existing efforts simply model the 2D visual dynamics, which is insufficient for robust manipulation when target tasks involve prominent depth-wise movement. To address this, we present a 3D dynamics-aware manipulation framework that seamlessly integrates 3D world modeling and policy learning. Three self-supervised learning tasks (current depth estimation, future RGB-D prediction, 3D flow prediction) are introduced within our framework, which complement each other and endow the policy model with 3D foresight. Extensive experiments on simulation and the real world show that 3D foresight can greatly boost the performance of manipulation policies without sacrificing inference speed. Code is available at https://github.com/Stardust-hyx/3D-Foresight.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑