arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1809.10253cs.LGcs.AIcs.ROstat.ML
收藏

通过学习可组合机器人技能来缩放仿真到真实的转换

Scaling simulation-to-real transfer by learning composable robot skills

University of Southern California(南加利福尼亚大学);Google Brain(谷歌大脑)

Ryan Julian, Eric Heiden, Zhanpeng He, Hejia Zhang, Stefan Schaal, Joseph J. Lim, Gaurav Sukhatme, Karol Hausman

摘要:

我们提出了一种新的解决方案,模拟到真实的传输,这是建立在机器人技能分解的最新进展的问题。我们不是专注于最小化模拟与现实之间的差距,而是学习一组不同的策略,这些策略以一种易于重用的方式进行参数化。这种低层次技能的多样性和参数化使我们能够找到一种可转移的策略,能够使用不同技能的组合和变化来解决更复杂的高层次任务。特别是,我们首先使用模拟来共同学习一组低级别技能的策略,以及可用于组成它们的“技能嵌入”参数化。稍后,我们学习通过这种技能嵌入参数化来启动低级策略的高级策略。高级策略编码了如何以及何时一起重用低级技能以实现特定的高级任务。重要的是,我们的方法学习控制关节空间中的真实机器人,以在机器人上很少或没有时间的情况下实现这些高级任务,尽管低级策略可能无法完全从模拟转移到真实,并且低级技能没有在高级任务的任何示例上进行训练。我们说明了我们的方法使用信息模拟实验的原则。然后,我们验证了它的实用性,真正的机器人问题的学习,转让,并组成自由空间和接触运动技能的索耶机器人只使用关节空间控制。我们实验了几种技术组成预先学习的技能,并发现我们的方法允许我们使用基于学习的方法和有效的基于搜索的规划,以实现高层次的任务,只使用预先学习的技能。

英文摘要:

We present a novel solution to the problem of simulation-to-real transfer, which builds on recent advances in robot skill decomposition. Rather than focusing on minimizing the simulation-reality gap, we learn a set of diverse policies that are parameterized in a way that makes them easily reusable. This diversity and parameterization of low-level skills allows us to find a transferable policy that is able to use combinations and variations of different skills to solve more complex, high-level tasks. In particular, we first use simulation to jointly learn a policy for a set of low-level skills, and a "skill embedding" parameterization which can be used to compose them. Later, we learn high-level policies which actuate the low-level policies via this skill embedding parameterization. The high-level policies encode how and when to reuse the low-level skills together to achieve specific high-level tasks. Importantly, our method learns to control a real robot in joint-space to achieve these high-level tasks with little or no on-robot time, despite the fact that the low-level policies may not be perfectly transferable from simulation to real, and that the low-level skills were not trained on any examples of high-level tasks. We illustrate the principles of our method using informative simulation experiments. We then verify its usefulness for real robotics problems by learning, transferring, and composing free-space and contact motion skills on a Sawyer robot using only joint-space control. We experiment with several techniques for composing pre-learned skills, and find that our method allows us to use both learning-based approaches and efficient search-based planning to achieve high-level tasks using only pre-learned skills.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑