加速与扩展MPC引导的强化学习在类人机器人行走与操作中的应用
Accelerating and Scaling MPC-Guided Reinforcement Learning for Humanoid Locomotion and Manipulation
AI总结：
本文提出了一种基于质心动力学MPC奖励的MPC-RL框架，并开发了并行批处理GPU求解器π^nMPC，以高效实现类人机器人的行走与操作技能。
摘要：
在类人运动控制中，模型预测控制（MPC）提供基于物理的预测和约束处理，而强化学习（RL）通过大规模仿真实现鲁棒的全身技能。然而，在RL内部使用MPC通常需要耗时的问题构建或过高的训练开销，使得此类框架在实践中难以证明其合理性。本文研究了训练时高效的MPC引导方法用于类人机器人行走与操作，称为MPC-RL。我们引入了一种基于质心动力学的MPC奖励公式，在训练时利用MPC轨迹的引导。为了在大规模并行RL中实现这一点，我们开发了π^nMPC，一种并行时域且无需构建的批处理GPU MPC求解器，它直接操作时变动力学以避免高内存使用和预编译。通过多种对比研究和硬件验证，我们发现MPC-RL在行走和操作技能上实现了优越的性能。代码库可在https://github.com/junhengl/mpc-rl获取。
英文摘要：
In humanoid motion control, model predictive control (MPC) offers physically grounded prediction and constraint handling, while reinforcement learning (RL) enables robust whole-body skills through large-scale simulation. However, using MPC inside RL often requires time-consuming problem construction or excessive training overhead, making such frameworks difficult to justify in practice. This work studies efficient training-time MPC guidance for humanoid locomotion and manipulation, termed MPC-RL. We introduce a centroidal-dynamics MPC reward formulation that leverages guidance from MPC trajectories in training time. To make this practical in massively parallel RL, we develop $π^n$MPC, a parallel-in-horizon and construction-free batched GPU MPC solver that operates directly on time-varying dynamics to avoid high memory usage and pre-compilation. Through a variety of comparative studies and hardware validations, we have found that MPC-RL achieves superior performance in locomotion and manipulation skills. The code base is available at https://github.com/junhengl/mpc-rl.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
