arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2109.13050cs.ROcs.LG
收藏

动作技能行为树中参数的学习

Learning of Parameters in Behavior Trees for Movement Skills

Lund University(隆德大学);Computer Technology Institute & Press “Diophantus”(“丢番图”计算机技术与出版研究所);University of Patras(帕特雷大学);Stanford University(斯坦福大学)

Matthias Mayr, Konstantinos Chatzilygeroudis, Faseeh Ahmad, Luigi Nardi, Volker Krueger

摘要:

强化学习(RL)是一个强大的数学框架,允许机器人通过试错来学习复杂的技能。尽管在许多应用中取得了许多成功,但RL算法仍然需要数千次试验才能收敛到高性能的策略,在学习时可能会产生危险的行为,并且优化的策略(通常建模为神经网络)在无法执行任务时几乎没有解释。由于这些原因,在工业环境中采用RL并不常见。另一方面,行为树(BT)可以提供策略表示,其a)支持模块化和可组合技能,b)允许容易地解释机器人动作,以及c)提供有利的低维参数空间。在本文中,我们提出了一种新的算法,可以在模拟中学习BT策略的参数,然后推广到物理机器人,而无需任何额外的训练。我们利用物理模拟器与我们的工作站的数字孪生,并优化与黑盒优化器的相关参数。我们展示了我们的方法的有效性与7自由度KUKA-iiwa机械手在一个任务,包括避障和接触丰富的插入(钉孔),其中我们的方法优于基线。

英文摘要:

Reinforcement Learning (RL) is a powerful mathematical framework that allows robots to learn complex skills by trial-and-error. Despite numerous successes in many applications, RL algorithms still require thousands of trials to converge to high-performing policies, can produce dangerous behaviors while learning, and the optimized policies (usually modeled as neural networks) give almost zero explanation when they fail to perform the task. For these reasons, the adoption of RL in industrial settings is not common. Behavior Trees (BTs), on the other hand, can provide a policy representation that a) supports modular and composable skills, b) allows for easy interpretation of the robot actions, and c) provides an advantageous low-dimensional parameter space. In this paper, we present a novel algorithm that can learn the parameters of a BT policy in simulation and then generalize to the physical robot without any additional training. We leverage a physical simulator with a digital twin of our workstation, and optimize the relevant parameters with a black-box optimizer. We showcase the efficacy of our method with a 7-DOF KUKA-iiwa manipulator in a task that includes obstacle avoidance and a contact-rich insertion (peg-in-hole), in which our method outperforms the baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑