发表机构
University of California, San Diego（加利福尼亚大学圣迭戈分校）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
PatternDex通过从人-物演示中学习交互模式，引导双臂灵巧手的强化学习，避免了具身差距，在ARCTIC数据集上平均成功率92.8%，并成功迁移到真实微波炉任务。
AI 中文摘要
本文中，我们开发了一种方法，使双臂灵巧手能够以高成功率操作铰接物体，而不会遭受具身差距的影响。我们观察到，手部运动与物体运动之间的相关性由物体而非手部决定，并且可以从人-物体演示中学习。基于这一观察，我们提出了PatternDex，一种学习这种相关性并将其表示为标记序列（称为交互模式）的方法。根据该模式，PatternDex估计适合目标机器人的手腕运动和接触点，然后训练一个利用这些估计作为引导的强化学习策略。由于引导适合目标具身，策略仅探索目标机器人能够执行的动作，因此实现了高成功率。PatternDex还只需要简单的微调即可训练新机器人，因为它可以重用学习到的交互模式。我们使用ARCTIC数据集中的演示，用双臂灵巧手评估了PatternDex。PatternDex在Allegro手上平均实现了92.8%的成功率，而最先进的基线为52.2%。此外，仅通过微调，它在其他三种机器人手上实现了超过70%的成功率。我们还验证了学习到的策略能够很好地迁移到打开微波炉的真实世界任务中。视频和额外结果可在该URL获取。
英文摘要
In this paper, we develop a method that enables bimanual dexterous hands to manipulate articulated objects with a high success rate without suffering from an embodiment gap. We observe that the correlation between hand motions and object motions is dictated by the object rather than the hands and can be learned from human-object demonstrations. Based on this observation, we propose PatternDex, a method that learns this correlation and represents it as a token sequence, which we call an interaction pattern. From this pattern, PatternDex estimates the wrist motions and contact points that fit the target robot, and then trains a reinforcement learning policy that exploits these estimates as guidance. Since the guidance fits the target embodiment, the policy explores only the actions that the target robot can execute and thus achieves high success rates. PatternDex also requires only simple fine-tuning to train a new robot, since it can reuse the learned interaction pattern. We evaluate PatternDex with bimanual dexterous hands on human demonstrations from the ARCTIC dataset. PatternDex achieves, on average, a 92.8% success rate with Allegro hands, while the state-of-the-art baseline achieves 52.2%. Also, it achieves success rates above 70% with three other robot hands after fine-tuning alone. Furthermore, we verify that the learned policy transfers well to a real-world task of opening a microwave. Videos and additional results are available at https://patterndex.github.io/PatternDex/
Comments8 pages, 5 figures, 3 tables. Project page: https://patterndex.github.io/PatternDex/
