通过RL增强的MPC学习敏捷运动与自适应行为
Learning Agile Locomotion and Adaptive Behaviors via RL-augmented MPC
AI总结：
本文提出结合RL与MPC的统一框架，通过将支撑足控制与摆动足反射相结合，实现四足机器人敏捷且鲁棒的无感知运动，并在多平台零样本迁移与重载下表现优异。
摘要：
在腿式机器人领域，自适应行为包括自适应平衡和自适应摆动足反射。自适应平衡用于抵抗机器人受到的扰动，而自适应摆动足反射则帮助机器人在复杂地形中行进且避免足部卡陷。本文通过结合RL和MPC，成功将这两方面的自适应行为引入四足运动，同时提升了无感知（盲）腿式运动的鲁棒性和敏捷性。这种结合利用了MPC在预测能力上的优势以及RL从过往经验中学习的擅长之处。与传统运动控制将支撑足控制和摆动足轨迹分离不同，我们的创新方法将二者统一，解决了它们之间缺乏同步的问题。我们贡献的核心是将支撑足控制与摆动足反射相结合，通过自适应行为提升运动的敏捷性和鲁棒性。该方法的一大标志是通过摆动足反射实现鲁棒的盲爬楼梯。此外，我们特意将学习模块设计为适用于不同机器人平台的通用插件。我们在Unitree A1机器人上训练了策略并实现了该方法，取得了令人瞩目的结果：峰值转向速率为8.5 rad/s，峰值奔跑速度为3 m/s，以及以2.5 m/s的速度进行转向。值得注意的是，该框架还使机器人在承受10 kg（即其体重的83%）的意外载荷时仍能保持稳定运动。我们进一步展示了同一策略的泛化性和鲁棒性，实现了向Go1和AlienGo等不同机器人平台的零样本迁移以进行载重。代码已在https://github.com/DRCL-USC/RL_augmented_MPC.git开源供研究社区使用。
英文摘要：
In the context of legged robots, adaptive behavior involves adaptive balancing and adaptive swing foot reflection. While adaptive balancing counteracts perturbations to the robot, adaptive swing foot reflection helps the robot to navigate intricate terrains without foot entrapment. In this paper, we manage to bring both aspects of adaptive behavior to quadruped locomotion by combining RL and MPC while improving the robustness and agility of blind legged locomotion. This integration leverages MPC's strength in predictive capabilities and RL's adeptness in drawing from past experiences. Unlike traditional locomotion controls that separate stance foot control and swing foot trajectory, our innovative approach unifies them, addressing their lack of synchronization. At the heart of our contribution is the synthesis of stance foot control with swing foot reflection, improving agility and robustness in locomotion with adaptive behavior. A hallmark of our approach is robust blind stair climbing through swing foot reflection. Moreover, we intentionally designed the learning module as a general plugin for different robot platforms. We trained the policy and implemented our approach on the Unitree A1 robot, achieving impressive results: a peak turn rate of 8.5 rad/s, a peak running speed of 3 m/s, and steering at a speed of 2.5 m/s. Remarkably, this framework also allows the robot to maintain stable locomotion while bearing an unexpected load of 10 kg, or 83\% of its body mass. We further demonstrate the generalizability and robustness of the same policy where it realizes zero-shot transfer to different robot platforms like Go1 and AlienGo robots for load carrying. Code is made available for the use of the research community at https://github.com/DRCL-USC/RL_augmented_MPC.git
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
