DextrAH-RGB：使用灵巧手抓取任何物体的视觉运动策略
DextrAH-RGB: Visuomotor Policies to Grasp Anything with Dexterous Hands
浏览论文内容
中文总结 AI 辅助
本文提出DextrAH-RGB系统，通过强化学习训练特权织物引导策略并蒸馏为基于RGB的策略，实现端到端灵巧抓取，在真实世界中对未见物体展现出强泛化能力与鲁棒的sim2real迁移。
中文摘要 AI 辅助
对于灵巧机器人而言，最重要的技能之一是抓取各种不同的物体，但这极具挑战性。许多先前的工作在速度、泛化能力或对深度图和物体位姿的依赖方面存在局限。在本文中，我们介绍了DextrAH-RGB，这是一个能够基于RGB图像输入端到端执行灵巧臂手抓取的系统。我们通过强化学习在仿真中训练了一个特权织物引导策略（FGP），该策略作用于几何织物控制器，以灵巧地抓取各种物体。然后，我们使用逼真的平铺渲染技术，严格在仿真中将这个特权FGP蒸馏为基于RGB的FGP。据我们所知，这是首个能够展示端到端基于RGB的策略在复杂、动态、富接触任务（如灵巧抓取）中实现鲁棒sim2real迁移的工作。DextrAH-RGB与基于深度的灵巧抓取策略相媲美，并在真实世界中泛化到具有未见几何形状、纹理和光照条件的新颖物体。
英文摘要
One of the most important, yet challenging, skills for a dexterous robot is grasping a diverse range of objects. Much of the prior work has been limited by speed, generality, or reliance on depth maps and object poses. In this paper, we introduce DextrAH-RGB, a system that can perform dexterous arm-hand grasping end-to-end from RGB image input. We train a privileged fabric-guided policy (FGP) in simulation through reinforcement learning that acts on a geometric fabric controller to dexterously grasp a wide variety of objects. We then distill this privileged FGP into a RGB-based FGP strictly in simulation using photorealistic tiled rendering. To our knowledge, this is the first work that is able to demonstrate robust sim2real transfer of an end2end RGB-based policy for complex, dynamic, contact-rich tasks such as dexterous grasping. DextrAH-RGB is competitive with depth-based dexterous grasping policies, and generalizes to novel objects with unseen geometry, texture, and lighting conditions in the real world. Videos of our system grasping a diverse range of unseen objects are available at \url{https://dextrah-rgb.github.io/}.
发表机构
- NVIDIA Corporation（英伟达公司）
- University of California, Berkeley（加州大学伯克利分校）
机构由 AI 辅助整理，请以论文原文为准。
