arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.12601cs.ROcs.AIcs.CVcs.LG
收藏

R3M:机器人操作的通用视觉表示

R3M: A Universal Visual Representation for Robot Manipulation

Stanford University(斯坦福大学);Meta AI

Suraj Nair, Aravind Rajeswaran, Vikash Kumar, Chelsea Finn, Abhinav Gupta

摘要:

我们研究了在不同的人类视频数据上预训练的视觉表示如何能够实现下游机器人操作任务的数据高效学习。具体地说,我们使用Ego4D人类视频数据集使用时间对比学习,视频语言对齐和L1惩罚的组合来预训练视觉表示,以鼓励稀疏和紧凑的表示。所得到的表示,R3M,可以被用作下游策略学习的冻结感知模块。在一套12个模拟机器人操作任务中,我们发现,与从头开始训练相比,R3M将任务成功率提高了20%以上,与CLIP和MoCo等最先进的视觉表示相比,提高了10%以上。此外,R3M还能让Franka Panda的手臂在一个真实的、杂乱的公寓里学习一系列操作任务,只需20次演示。代码和预训练模型可在https://tinyurl.com/robotr3m上获得。

英文摘要:

We study how visual representations pre-trained on diverse human video data can enable data-efficient learning of downstream robotic manipulation tasks. Concretely, we pre-train a visual representation using the Ego4D human video dataset using a combination of time-contrastive learning, video-language alignment, and an L1 penalty to encourage sparse and compact representations. The resulting representation, R3M, can be used as a frozen perception module for downstream policy learning. Across a suite of 12 simulated robot manipulation tasks, we find that R3M improves task success by over 20% compared to training from scratch and by over 10% compared to state-of-the-art visual representations like CLIP and MoCo. Furthermore, R3M enables a Franka Emika Panda arm to learn a range of manipulation tasks in a real, cluttered apartment given just 20 demonstrations. Code and pre-trained models are available at https://tinyurl.com/robotr3m.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑