arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2301.09766cs.ROcs.AIcs.LG
收藏

灵巧操作的约束强化学习

Constrained Reinforcement Learning for Dexterous Manipulation

Georgia Institute of Technology(佐治亚理工学院)

Abhineet Jain, Jack Kolb, Harish Ravichandar

摘要:

现有的灵巧操作的学习方法使用演示或与环境的交互来训练黑盒神经网络,该黑盒神经网络对机器人如何学习技能或它将如何执行后训练提供很少的控制。这些方法在物理平台上实现时会带来重大挑战,因为在训练的初始阶段,机器人的行为可能不稳定,并可能对其自身的硬件、环境或附近的任何人造成伤害。解决这些限制的一种潜在方法是在学习过程中添加约束,以限制和指导机器人在训练和推出过程中的行为。受其他领域中约束方法的成功启发,我们研究了将基于位置的约束添加到24自由度机器人手学习中以使用约束策略优化执行对象重新定位的效果。我们发现,一个简单的几何约束可以确保机器人学会向物体移动比没有约束。此外,具有此约束的训练需要与其无约束对应物类似数量的样本来掌握技能。这些发现揭示了简单的约束如何帮助机器人快速实现明智和安全的行为,并缓解围绕硬件部署的担忧。我们还调查了这些约束的严格性的影响,并报告了研究结果,这些结果提供了不同程度的严格性如何影响学习成果的见解。我们的代码可在https://github.com/GT-STAR-Lab/constrained-rl-dexterous-manipulation上获得。

英文摘要:

Existing learning approaches to dexterous manipulation use demonstrations or interactions with the environment to train black-box neural networks that provide little control over how the robot learns the skills or how it would perform post training. These approaches pose significant challenges when implemented on physical platforms given that, during initial stages of training, the robot's behavior could be erratic and potentially harmful to its own hardware, the environment, or any humans in the vicinity. A potential way to address these limitations is to add constraints during learning that restrict and guide the robot's behavior during training as well as roll outs. Inspired by the success of constrained approaches in other domains, we investigate the effects of adding position-based constraints to a 24-DOF robot hand learning to perform object relocation using Constrained Policy Optimization. We find that a simple geometric constraint can ensure the robot learns to move towards the object sooner than without constraints. Further, training with this constraint requires a similar number of samples as its unconstrained counterpart to master the skill. These findings shed light on how simple constraints can help robots achieve sensible and safe behavior quickly and ease concerns surrounding hardware deployment. We also investigate the effects of the strictness of these constraints and report findings that provide insights into how different degrees of strictness affect learning outcomes. Our code is available at https://github.com/GT-STAR-Lab/constrained-rl-dexterous-manipulation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑