基于深度感知预训练的视觉机器人操作
Visual Robotic Manipulation with Depth-Aware Pretraining
AI总结：
本文提出深度感知预训练方法DPR，利用大规模3D数据以自监督对比学习增强RGB视觉骨干的3D场景表征能力，并引入本体感知注入策略网络，在仿真与真实机器人操作任务中提升对未见物体和环境的泛化性能。
摘要：
近年来关于视觉表征学习的研究已被证明对机器人操作任务十分有效。然而，现有大多数工作仅在2D图像或第一人称视频上预训练视觉骨干网络，忽略了机器人需要在3D空间中学习行动这一事实，而这一点很难从2D观测中习得。本文研究了利用公开可用的大规模3D数据对视觉骨干网络进行预训练以改进操作策略学习的有效性。我们提出的方法名为面向机器人操作的深度感知预训练（Depth-aware Pretraining for Robotics，DPR），使仅使用RGB输入的骨干网络能够通过自监督对比学习来学习3D场景表征，其中深度信息作为辅助知识。在操作策略学习和推理阶段不需要任何3D信息，使我们的模型在3D空间操作中同时具备高效性和有效性。此外，我们引入了一种将机器人本体感知注入策略网络的新方法，使操作模型具有鲁棒性和泛化能力。我们在实验中证明，所提出的框架在仿真机器人和真实机器人上的多种机器人任务中，均能提升对未见物体和视觉环境的性能表现。
英文摘要：
Recent work on visual representation learning has shown to be efficient for robotic manipulation tasks. However, most existing works pretrained the visual backbone solely on 2D images or egocentric videos, ignoring the fact that robots learn to act in 3D space, which is hard to learn from 2D observation. In this paper, we examine the effectiveness of pretraining for vision backbone with public-available large-scale 3D data to improve manipulation policy learning. Our method, namely Depth-aware Pretraining for Robotics (DPR), enables an RGB-only backbone to learn 3D scene representations from self-supervised contrastive learning, where depth information serves as auxiliary knowledge. No 3D information is necessary during manipulation policy learning and inference, making our model enjoy both efficiency and effectiveness in 3D space manipulation. Furthermore, we introduce a new way to inject robots' proprioception into the policy networks that makes the manipulation model robust and generalizable. We demonstrate in experiments that our proposed framework improves performance on unseen objects and visual environments for various robotics tasks on both simulated and real robots.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
