arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.04868eess.SYcs.AIcs.SY
收藏

面向隐性平衡的模型预测控制引导强化学习

Model Predictive Control-Guided Reinforcement Learning for Implicit Balancing

IDLab, Ghent University–imec(IDLab、根特大学–imec);Delft University of Technology, Faculty of Technology, Policy, and Management(代尔夫特理工大学、技术政策与管理学院)

Seyed Soroush Karimi Madahi, Kenneth Bruninx, Bert Claessens, Chris Develder

AI总结:

针对欧洲隐性平衡场景中MPC成本高、无模型RL训练数据需求大的问题,提出MPC引导的RL方法,兼具预测融合能力与快速推理优势,在比利时电池控制场景下套利收益显著优于单一方法。

摘要:

在欧洲,以盈利为目的的平衡责任方可以在实时运行中偏离其日前申报量,以协助输电系统运营商维持供需平衡。利用这些隐性平衡策略的模型预测控制(MPC)策略能够捕捉套利机会,但无法准确刻画欧洲不平衡市场中的价格形成过程,且面临高昂的计算成本。无模型强化学习(RL)方法执行速度快,但需要数据密集型训练,且通常依赖实时和历史数据进行决策。本文提出一种MPC引导的RL方法,结合了MPC与RL二者的互补优势。该方法能够像MPC一样有效地将预测结果纳入决策过程,同时保持RL的快速推理能力。研究使用2023年比利时平衡数据,在隐性平衡电池控制问题上评估了所提方法的性能。首先,本文从多个角度分析了独立使用的最先进RL与MPC方法的性能,以凸显二者各自的优势与局限性。随后,结果表明所提的MPC引导RL方法相较于独立RL和独立MPC,套利收益分别提升16.15%和54.36%。

英文摘要:

In Europe, profit-seeking balance responsible parties can deviate in real time from their day-ahead nominations to assist transmission system operators in maintaining the supply-demand balance. Model predictive control (MPC) strategies to exploit these implicit balancing strategies capture arbitrage opportunities, but fail to accurately capture the price-formation process in the European imbalance markets and face high computational costs. Model-free reinforcement learning (RL) methods are fast to execute, but require data-intensive training and usually rely on real-time and historical data for decision-making. This paper proposes an MPC-guided RL method that combines the complementary strengths of both MPC and RL. The proposed method can effectively incorporate forecasts into the decision-making process (as in MPC), while maintaining the fast inference capability of RL. The performance of the proposed method is evaluated on the implicit balancing battery control problem using Belgian balancing data from 2023. First, we analyze the performance of the standalone state-of-the-art RL and MPC methods from various angles, to highlight their individual strengths and limitations. Next, we show an arbitrage profit benefit of the proposed MPC-guided RL method of 16.15% and 54.36%, compared to standalone RL and MPC.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑