arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2503.08950cs.ROcs.AI
收藏

FP3:一种用于机器人操作的三维基础策略

FP3: A 3D Foundation Policy for Robotic Manipulation

Shanghai AI Laboratory(上海人工智能实验室);Shanghai Qi Zhi Institute(上海期智研究院)

Rujia Yang, Geng Chen, Chuan Wen, Yang Gao

AI总结:

本文提出首个大规模三维机器人操作基础策略模型FP3,基于扩散Transformer在6万条点云轨迹上预训练,仅需80次演示即可在新环境中以超90%成功率学习新任务。

摘要:

继在自然语言处理和计算机视觉领域取得成功之后,在大规模多任务数据集上预训练的基础模型也在机器人领域展现出巨大潜力。然而,现有大多数机器人基础模型仅依赖二维图像观测,忽略了三维几何信息,而三维几何信息对于机器人感知和推理三维世界至关重要。本文提出了FP3,这是首个面向机器人操作的大规模三维基础策略模型。FP3建立在可扩展的扩散Transformer架构之上,并在包含6万条轨迹的点云观测数据上进行预训练。凭借模型设计和多样化的预训练数据,FP3能够针对下游任务进行高效微调,同时展现出强大的泛化能力。在真实机器人上的实验表明,仅需80次演示,FP3就能在具有未见物体的新环境中以超过90%的成功率学习新任务,显著超越现有机器人基础模型。

英文摘要:

Following its success in natural language processing and computer vision, foundation models that are pre-trained on large-scale multi-task datasets have also shown great potential in robotics. However, most existing robot foundation models rely solely on 2D image observations, ignoring 3D geometric information, which is essential for robots to perceive and reason about the 3D world. In this paper, we introduce FP3, a first large-scale 3D foundation policy model for robotic manipulation. FP3 builds on a scalable diffusion transformer architecture and is pre-trained on 60k trajectories with point cloud observations. With the model design and diverse pre-training data, FP3 can be efficiently fine-tuned for downstream tasks while exhibiting strong generalization capabilities. Experiments on real robots demonstrate that with only 80 demonstrations, FP3 is able to learn a new task with over 90% success rates in novel environments with unseen objects, significantly surpassing existing robot foundation models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑