不仅是奖励还有约束：在足式机器人运动中的应用
Not Only Rewards But Also Constraints: Applications on Legged Robot Locomotion
AI总结：
提出一种结合奖励与约束的强化学习框架，通过两种约束类型和高效策略优化算法训练足式机器人运动控制器，大幅减少奖励工程量并提升工程直观性。
摘要：
早期几项研究通过使用神经网络设计控制器并采用无模型强化学习进行训练，在复杂机器人系统中展现了出色的控制性能。然而，这些具有自然运动风格和高任务性能的优秀控制器是通过大量的奖励工程开发出来的，这是一个极其费力且耗时的过程，需要设计众多奖励项并确定合适的奖励系数。在本研究中，我们提出了一种新颖的强化学习框架，用于训练由奖励和约束共同组成的复杂机器人系统神经网络控制器。为了让工程师能够适当地将其意图反映到约束中，并以最小的计算开销进行处理，我们提出了两种约束类型和一种高效的策略优化算法。该学习框架被应用于训练具有不同形态和物理属性的几款足式机器人的运动控制器，以穿越复杂地形。大量的仿真和真实世界实验表明，仅需调整单个奖励系数，即可在显著减少奖励工程的情况下训练出高性能的控制器。此外，得益于约束的可解释性和泛化能力，可以采用更直接、更直观的工程流程。总结视频请见 https://youtu.be/KAlm3yskhvM。
英文摘要：
Several earlier studies have shown impressive control performance in complex robotic systems by designing the controller using a neural network and training it with model-free reinforcement learning. However, these outstanding controllers with natural motion style and high task performance are developed through extensive reward engineering, which is a highly laborious and time-consuming process of designing numerous reward terms and determining suitable reward coefficients. In this work, we propose a novel reinforcement learning framework for training neural network controllers for complex robotic systems consisting of both rewards and constraints. To let the engineers appropriately reflect their intent to constraints and handle them with minimal computation overhead, two constraint types and an efficient policy optimization algorithm are suggested. The learning framework is applied to train locomotion controllers for several legged robots with different morphology and physical attributes to traverse challenging terrains. Extensive simulation and real-world experiments demonstrate that performant controllers can be trained with significantly less reward engineering, by tuning only a single reward coefficient. Furthermore, a more straightforward and intuitive engineering process can be utilized, thanks to the interpretability and generalizability of constraints. The summary video is available at https://youtu.be/KAlm3yskhvM.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
