arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.03354eess.SYcs.SY
收藏

关于结合强化学习与模型预测控制并实现运行时改进的架构

On Architectures for Combining Reinforcement Learning and Model Predictive Control with Runtime Improvements

Xiaolong Jia, Nikhil Bajaj

AI总结:

本文提出Warm Start RL与RLMPC两种融合NNMPC和强化学习的架构,并配合下采样降维,在旋转倒立摆上实现超过99%的运行时缩减及不确定性下更好的跟踪性能。

摘要:

模型预测控制(MPC)面临计算需求以及模型不准确导致的性能下降问题。我们提出两种将神经网络近似MPC(NNMPC)与强化学习(RL)相结合的架构。第一种是Warm Start RL,使用预训练的NNMPC权重初始化RL actor。第二种是RLMPC,利用RL为NNMPC输出生成校正残差。我们还引入一种下采样方法,在保持性能的同时降低NNMPC输入维度。在旋转倒立摆上的评估表明,与传统MPC相比,两种架构均实现超过99%的运行时缩减,并在模型不确定性下提升跟踪性能;其中RL+MPC根据参考幅值实现11%至40%的成本降低。

英文摘要:

Model Predictive Control (MPC) faces computational demands and performance degradation from model inaccuracies. We propose two architectures combining Neural Network-approximated MPC (NNMPC) with Reinforcement Learning (RL). The first, Warm Start RL, initializes the RL actor with pre-trained NNMPC weights. The second, RLMPC, uses RL to generate corrective residuals for NNMPC outputs. We introduce a downsampling method reducing NNMPC input dimensions while maintaining performance. Evaluated on a rotary inverted pendulum, both architectures demonstrate runtime reductions exceeding 99% compared to traditional MPC while improving tracking performance under model uncertainties, with RL+MPC achieving 11-40% cost reduction depending on reference amplitude.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑