arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1812.03201cs.ROcs.LG
收藏

机器人控制中的残差强化学习

Residual Reinforcement Learning for Robot Control

Siemens Corporation(西门子公司);University of California, Berkeley(加州大学伯克利分校);Hamburg University of Technology(汉堡工业大学)

Tobias Johannink, Shikhar Bahl, Ashvin Nair, Jianlan Luo, Avinash Kumar, Matthias Loskyll, Juan Aparicio Ojea, Eugen Solowjow, Sergey Levine

摘要:

传统的反馈控制方法可以非常有效地解决各种类型的机器人控制问题,通过捕获的结构与明确的模型,如刚体运动方程。然而,现代制造中的许多控制问题涉及接触和摩擦,这很难用一阶物理模型来描述。因此,将控制设计方法应用于这类问题通常会导致脆弱和不准确的控制器,必须手动调整部署。强化学习(RL)方法已被证明能够从与环境的交互中学习连续的机器人控制器,即使是包括摩擦和接触在内的问题。在本文中,我们研究如何解决困难的控制问题,在现实世界中,通过分解成一个部分,是有效地解决了传统的反馈控制方法,和剩余的解决与RL。最终的控制策略是两种控制信号的叠加。我们证明了我们的方法,通过训练代理成功地执行一个现实世界的块组装任务,涉及接触和不稳定的对象。

英文摘要:

Conventional feedback control methods can solve various types of robot control problems very efficiently by capturing the structure with explicit models, such as rigid body equations of motion. However, many control problems in modern manufacturing deal with contacts and friction, which are difficult to capture with first-order physical modeling. Hence, applying control design methodologies to these kinds of problems often results in brittle and inaccurate controllers, which have to be manually tuned for deployment. Reinforcement learning (RL) methods have been demonstrated to be capable of learning continuous robot controllers from interactions with the environment, even for problems that include friction and contacts. In this paper, we study how we can solve difficult control problems in the real world by decomposing them into a part that is solved efficiently by conventional feedback control methods, and the residual which is solved with RL. The final control policy is a superposition of both control signals. We demonstrate our approach by training an agent to successfully perform a real-world block assembly task involving contacts and unstable objects.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑