通过自监督初始化学习实现更快的模型预测控制
Faster Model Predictive Control via Self-Supervised Initialization Learning
AI总结:
本文提出一种热启动框架,通过行为克隆和强化学习训练策略为MPC求解器生成高质量初始猜测,以最小化优化时间,实验证明该方法不仅加速了确定性和采样类MPC,还提升了整体控制性能。
摘要:
模型预测控制(MPC)通过在有限时域上优化控制输出序列,被广泛应用于机器人控制。MPC的计算方法包括确定性方法(如iLQR和COBYLA)以及基于采样的方法(如MPPI和CEM)。然而,复杂的系统动力学和非凸或不可微的代价项通常会导致难以接受的优化时间,从而限制了现实世界的部署。先前加速MPC的努力存在以下局限性:(i)重用先前的解在状态急剧变化时失效;(ii)纯模仿学习不直接针对计算效率,且受训练数据次优性的影响。为了解决这些问题,我们提出了一个热启动框架,该框架学习一个策略来为MPC求解器生成高质量的初始猜测。该策略首先通过行为克隆从专家MPC rollout中训练,然后通过强化学习进行在线微调,以直接最小化MPC优化时间。我们经验性地验证了我们的方法改进了确定性和基于采样的MPC方法,在Formula 1赛道路径跟踪领域,对于确定性MPC实现了高达21.6%的优化速度提升和34.1%的跟踪精度提升;在障碍物密集的导航领域,对于基于采样的MPC,安全性提高了100%,路径效率提高了12.8%,转向平滑度提高了7.2%。这些结果表明,我们的框架不仅加速了MPC,还提高了整体控制性能。此外,它还可以应用于更广泛的受益于良好初始猜测的控制算法。
英文摘要:
Model Predictive Control (MPC) is widely used in robot control by optimizing a sequence of control outputs over a finite-horizon. Computational approaches for MPC include deterministic methods (e.g., iLQR and COBYLA), as well as sampling-based methods (e.g., MPPI and CEM). However, complex system dynamics and non-convex or non-differentiable cost terms often lead to prohibitive optimization times that limit real-world deployment. Prior efforts to accelerate MPC have limitations on: (i) reusing previous solutions fails under sharp state changes and (ii) pure imitation learning does not target compute efficiency directly and suffers from suboptimality in the training data. To address these, We propose a warm-start framework that learns a policy to generate high-quality initial guesses for MPC solver. The policy is first trained via behavior cloning from expert MPC rollouts and then fine-tuned online with reinforcement learning to directly minimize MPC optimization time. We empirically validate that our approach improves both deterministic and sampling-based MPC methods, achieving up to 21.6% faster optimization and 34.1% more tracking accuracy for deterministic MPC in Formula 1 track path-tracking domain, and improving safety by 100%, path efficiency by 12.8%, and steering smoothness by 7.2% for sampling-based MPC in obstacle-rich navigation domain. These results demonstrate that our framework not only accelerates MPC but also improves overall control performance. Furthermore, it can be applied to a broader range of control algorithms that benefit from good initial guesses.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。