使用大规模并行深度强化学习在几分钟内学会走路
Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning
摘要:
在这项工作中,我们提出并研究了一个训练设置,实现了快速的政策生成现实世界的机器人任务,通过使用大规模的并行在一个工作站GPU。我们分析和讨论了不同的训练算法组件在大规模并行制度的最终政策的性能和训练时间的影响。此外,我们提出了一个新颖的游戏启发的课程,非常适合与成千上万的模拟机器人并行训练。我们通过训练四足机器人ANYmal在具有挑战性的地形上行走来评估这种方法。平行的方法允许在四分钟内训练平坦地形的策略,在二十分钟内训练不平坦地形的策略。与以前的工作相比,这代表了多个数量级的加速。最后,我们将策略转移到真实的机器人来验证该方法。我们开源了我们的训练代码,以帮助加速在学习腿部运动领域的进一步研究。
英文摘要:
In this work, we present and study a training set-up that achieves fast policy generation for real-world robotic tasks by using massive parallelism on a single workstation GPU. We analyze and discuss the impact of different training algorithm components in the massively parallel regime on the final policy performance and training times. In addition, we present a novel game-inspired curriculum that is well suited for training with thousands of simulated robots in parallel. We evaluate the approach by training the quadrupedal robot ANYmal to walk on challenging terrain. The parallel approach allows training policies for flat terrain in under four minutes, and in twenty minutes for uneven terrain. This represents a speedup of multiple orders of magnitude compared to previous work. Finally, we transfer the policies to the real robot to validate the approach. We open-source our training code to help accelerate further research in the field of learned legged locomotion.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。