arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1904.04152eess.SYcs.SY
收藏

基于强化学习的数据驱动经济非线性模型预测控制

Data-driven Economic NMPC using Reinforcement Learning

Chalmers University of Technology(查尔姆斯理工大学);IMT School for Advanced Studies Lucca(卢卡高等研究院)

Sébastien Gros, Mario Zanon

AI总结:

本文提出了一种利用强化学习的数据驱动经济非线性模型预测控制方法,展示了即使在使用错误模型的情况下,(E)NMPC方案也能提供真实系统的最优策略,并探讨了其在强化学习中的应用及与耗散性概念的联系。

摘要:

强化学习(RL)是一种强大的工具,用于执行数据驱动的最优控制,而无需依赖系统的模型。然而,RL在提供结果控制方案行为的硬性保证方面存在困难。相比之下,非线性模型预测控制(NMPC)和经济NMPC(ENMPC)是复杂系统闭环最优控制的标准工具,具有丰富的理论来评估其闭环行为。不幸的是,(E)NMPC的性能依赖于底层模型的质量。在本文中,我们证明,即使使用错误的模型,(E)NMPC方案也可以提供真实系统的最优策略。这一结果也适用于具有随机动态的真实系统。这意味着ENMPC可以作为一种新的函数近似器用于RL。此外,我们探讨了这些结果在ENMPC中的应用,并正式将其与耗散性概念联系起来,后者对ENMPC的稳定性至关重要。最后,我们详细说明了如何利用这些结果来部署经典RL工具来调整(E)NMPC方案。我们这些工具应用于经典的线性MPC设置和标准的ENMPC文献中的非线性示例。

英文摘要:

Reinforcement Learning (RL) is a powerful tool to perform data-driven optimal control without relying on a model of the system. However, RL struggles to provide hard guarantees on the behavior of the resulting control scheme. In contrast, Nonlinear Model Predictive Control (NMPC) and Economic NMPC (ENMPC) are standard tools for the closed-loop optimal control of complex systems with constraints and limitations, and benefit from a rich theory to assess their closed-loop behavior. Unfortunately, the performance of (E)NMPC hinges on the quality of the model underlying the control scheme. In this paper, we show that an (E)NMPC scheme can be tuned to deliver the optimal policy of the real system even when using a wrong model. This result also holds for real systems having stochastic dynamics. This entails that ENMPC can be used as a new type of function approximator within RL. Furthermore, we investigate our results in the context of ENMPC and formally connect them to the concept of dissipativity, which is central for the ENMPC stability. Finally, we detail how these results can be used to deploy classic RL tools for tuning (E)NMPC schemes. We apply these tools on both a classical linear MPC setting and a standard nonlinear example from the ENMPC literature.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑