arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2004.10886cs.RO
收藏

面向接触丰富操作的稳定性保证强化学习

Stability-Guaranteed Reinforcement Learning for Contact-rich Manipulation

Royal Institute of Technology(瑞典皇家理工学院);ASEA Brown Boveri (ABB) Corporate Research(ABB集团企业研究院)

Shahbaz A. Khader, Hang Yin, Pietro Falco, Danica Kragic

摘要:

强化学习(RL)在接触丰富的操作任务中取得了相当大的成功,但它仍然落后于机器人控制理论的进步,如阻抗控制和稳定性保证。最近,可变阻抗控制(VIC)的概念被采用到RL中,并取得了令人鼓舞的结果。然而,更重要的稳定问题仍未得到解决。为了澄清稳定强化学习中的挑战,我们引入了术语“全时稳定性”,这明确意味着每一个可能的部署都将获得稳定性认证。我们的贡献是一个无模型RL方法,不仅采用VIC,但也实现了所有的时间稳定性。基于最近提出的稳定VIC控制器作为策略参数化,我们引入了一种新的策略搜索算法,该算法受到交叉熵方法的启发,本质上保证了稳定性。我们的实验研究证实了稳定性保证的可行性和实用性,并具有,据我们所知,第一次成功应用RL与所有的时间稳定的基准问题的钉孔。

英文摘要:

Reinforcement learning (RL) has had its fair share of success in contact-rich manipulation tasks but it still lags behind in benefiting from advances in robot control theory such as impedance control and stability guarantees. Recently, the concept of variable impedance control (VIC) was adopted into RL with encouraging results. However, the more important issue of stability remains unaddressed. To clarify the challenge in stable RL, we introduce the term all-the-time-stability that unambiguously means that every possible rollout will be stability certified. Our contribution is a model-free RL method that not only adopts VIC but also achieves all-the-time-stability. Building on a recently proposed stable VIC controller as the policy parameterization, we introduce a novel policy search algorithm that is inspired by Cross-Entropy Method and inherently guarantees stability. Our experimental studies confirm the feasibility and usefulness of stability guarantee and also features, to the best of our knowledge, the first successful application of RL with all-the-time-stability on the benchmark problem of peg-in-hole.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑