arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.03486cs.RO
收藏

基于采样的探索用于灵巧操作的强化学习

Sampling-based Exploration for Reinforcement Learning of Dexterous Manipulation

Columbia University(哥伦比亚大学)

Gagan Khandate, Siqi Shang, Eric T. Chang, Tristan Luca Saidi, Yang Liu, Seth Matthew Dennis, Johnson Adams, Matei Ciocarlie

AI总结:

本文提出利用非完整快速扩展随机树算法进行基于采样的探索,生成重置分布以训练无模型强化学习控制策略,解决了高维状态空间探索难题,实现了复杂物体的无支撑灵巧操作并有效迁移至真实机器人。

摘要:

在本文中,我们提出了一种实现复杂物体灵巧操作的新方法,同时在不使用被动支撑面的情况下稳固物体。我们认为,在强化学习框架中训练此类策略的一个关键难点在于问题状态空间的探索难度,因为该空间的可访问区域在高维空间流形上形成了复杂的结构。为应对这一挑战,我们使用了两个版本的非完整快速扩展随机树算法;一个版本更具通用性,但需要显式使用环境的转移函数,而第二个版本利用特定于操作的运动学约束以获得更好的样本效率。在这两种情况下,我们均使用通过基于采样的探索发现的状态来生成重置分布,从而能够通过无模型强化学习在完整动力学约束下训练控制策略。我们表明,这些策略在比以往展示的难度更高的操作问题上具有有效性,并且能有效迁移到真实机器人上。真实机械手演示的视频可在项目网站上找到:https://sbrl.cs.columbia.edu/

英文摘要:

In this paper, we present a novel method for achieving dexterous manipulation of complex objects, while simultaneously securing the object without the use of passive support surfaces. We posit that a key difficulty for training such policies in a Reinforcement Learning framework is the difficulty of exploring the problem state space, as the accessible regions of this space form a complex structure along manifolds of a high-dimensional space. To address this challenge, we use two versions of the non-holonomic Rapidly-Exploring Random Trees algorithm; one version is more general, but requires explicit use of the environment's transition function, while the second version uses manipulation-specific kinematic constraints to attain better sample efficiency. In both cases, we use states found via sampling-based exploration to generate reset distributions that enable training control policies under full dynamic constraints via model-free Reinforcement Learning. We show that these policies are effective at manipulation problems of higher difficulty than previously shown, and also transfer effectively to real robots. Videos of the real-hand demonstrations can be found on the project website: https://sbrl.cs.columbia.edu/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑