DexRepNet++: 通过几何和空间手-物体表示学习灵巧的机器人操作
DexRepNet++: Learning Dexterous Robotic Manipulation with Geometric and Spatial Hand-Object Representations
AI总结：
DexRepNet++通过几何和空间手-物体表示提升机器人灵巧操作的泛化能力和效率。
摘要：
机器人灵巧操作是一个具有挑战性的问题，由于高自由度（DoFs）和多指机械手与物体之间复杂的接触。许多现有的深度强化学习（DRL）方法旨在提高高维输出动作空间中的样本效率。然而，现有工作往往忽略了在复杂输入空间中通过手-物体交互实现操作策略泛化时表示的作用。在本文中，我们提出了DexRep，一种新的手-物体交互表示，以捕捉物体表面特征和手与物体之间的空间关系，用于灵巧操作技能学习。基于DexRep，为三种灵巧操作任务学习了策略，即抓取、手内重定向和双臂手部交接，并进行了大量实验以验证有效性。在仿真中，对于抓取任务，使用40个物体学习的策略在超过5000个不同类别的未见过的物体上实现了87.9%的成功率，显著超越了使用数千个物体训练的现有工作；对于手内重定向和交接任务，策略也提升了现有手-物体表示的成功率和其他指标，提高了20%至40%。DexRep的抓取策略在多摄像头和单摄像头设置下部署到现实世界中，并展示了较小的仿真到现实差距。
英文摘要：
Robotic dexterous manipulation is a challenging problem due to high degrees of freedom (DoFs) and complex contacts of multi-fingered robotic hands. Many existing deep reinforcement learning (DRL) based methods aim at improving sample efficiency in high-dimensional output action spaces. However, existing works often overlook the role of representations in achieving generalization of a manipulation policy in the complex input space during the hand-object interaction. In this paper, we propose DexRep, a novel hand-object interaction representation to capture object surface features and spatial relations between hands and objects for dexterous manipulation skill learning. Based on DexRep, policies are learned for three dexterous manipulation tasks, i.e. grasping, in-hand reorientation, bimanual handover, and extensive experiments are conducted to verify the effectiveness. In simulation, for grasping, the policy learned with 40 objects achieves a success rate of 87.9% on more than 5000 unseen objects of diverse categories, significantly surpassing existing work trained with thousands of objects; for the in-hand reorientation and handover tasks, the policies also boost the success rates and other metrics of existing hand-object representations by 20% to 40%. The grasp policies with DexRep are deployed to the real world under multi-camera and single-camera setups and demonstrate a small sim-to-real gap.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
