使用以对象为中心的3D表征学习可泛化的操作策略
Learning Generalizable Manipulation Policies with Object-Centric 3D Representations
AI总结：
提出模仿学习方法GROOT，构建对背景和视角变化鲁棒的以对象为中心的3D表征，并结合transformer策略与分割对应模型，使视觉操作策略能泛化至新场景与新对象。
摘要：
我们引入了GROOT，这是一种通过以对象为中心和3D先验来学习鲁棒策略的模仿学习方法。GROOT构建的策略在基于视觉的操作中能够泛化到超出其初始训练条件之外的场景。它构建了对背景变化和相机视角具有鲁棒性的以对象为中心的3D表征，并使用基于transformer的策略对这些表征进行推理。此外，我们引入了一种分割对应模型，允许策略在测试时泛化到新对象。通过全面的实验，我们在模拟和真实环境中验证了GROOT策略对感知变化的鲁棒性。GROOT的性能在背景变化、相机视角偏移以及新对象实例出现等泛化方面表现卓越，而最先进的端到端学习方法和基于对象提议的方法均表现不足。我们还在真实机器人上广泛评估了GROOT策略，展示了其在非常剧烈的设置变化下的有效性。更多视频和模型详情可在附录和项目网站找到：https://ut-austin-rpl.github.io/GROOT。
英文摘要：
We introduce GROOT, an imitation learning method for learning robust policies with object-centric and 3D priors. GROOT builds policies that generalize beyond their initial training conditions for vision-based manipulation. It constructs object-centric 3D representations that are robust toward background changes and camera views and reason over these representations using a transformer-based policy. Furthermore, we introduce a segmentation correspondence model that allows policies to generalize to new objects at test time. Through comprehensive experiments, we validate the robustness of GROOT policies against perceptual variations in simulated and real-world environments. GROOT's performance excels in generalization over background changes, camera viewpoint shifts, and the presence of new object instances, whereas both state-of-the-art end-to-end learning methods and object proposal-based approaches fall short. We also extensively evaluate GROOT policies on real robots, where we demonstrate the efficacy under very wild changes in setup. More videos and model details can be found in the appendix and the project website: https://ut-austin-rpl.github.io/GROOT .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
