arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2305.05484eess.SYcs.SY
收藏

最优能源系统调度:一种约束感知的强化学习算法

Optimal Energy System Scheduling Using A Constraint-Aware Reinforcement Learning Algorithm

Hou Shengren, Pedro P. Vergara, Edgar Mauricio Salazar Duque, Peter Palensky

AI总结:

针对现有深度强化学习算法难以严格满足能源系统运行约束的问题,提出MIP-DQN算法,利用深度神经网络可表示为混合整数规划的特性,在动作空间强制执行所有约束,仿真验证其优于现有算法且误差更低。

摘要:

可再生能源分布式能源资源(DERs)的大规模集成本质上增加了能源系统的复杂性,尤其是在确定其运行调度方案时。深度强化学习(DRL)算法因其数据驱动和模型无关的特性而成为一种有前景的解决方案。然而,现有的DRL算法无法强制执行严格的运行约束(如功率平衡、爬坡上下限约束),限制了其在实际系统中的部署。为解决这一问题,本文提出了一种DRL算法(即MIP-DQN),该算法能够在动作空间中严格强制执行所有运行约束,确保所制定调度方案在实时运行中的可行性。这是通过利用深度神经网络(DNNs)的最新优化进展实现的,这些进展允许将DNN表示为混合整数规划(MIP)形式,从而能够进一步考虑任意动作空间约束。全面的数值仿真表明,所提出的算法优于现有最先进的DRL算法,在对比具有完美预测信息的数学规划公式求解得到的全局最优解(上界)时,获得了更低的误差;同时严格强制执行所有运行约束(即使在未见过的测试日)。

英文摘要:

The massive integration of renewable-based distributed energy resources (DERs) inherently increases the energy system's complexity, especially when it comes to defining its operational schedule. Deep reinforcement learning (DRL) algorithms arise as a promising solution due to their data-driven and model-free features. However, current DRL algorithms fail to enforce rigorous operational constraints (e.g., power balance, ramping up or down constraints) limiting their implementation in real systems. To overcome this, in this paper, a DRL algorithm (namely MIP-DQN) is proposed, capable of \textit{strictly} enforcing all operational constraints in the action space, ensuring the feasibility of the defined schedule in real-time operation. This is done by leveraging recent optimization advances for deep neural networks (DNNs) that allow their representation as a MIP formulation, enabling further consideration of any action space constraints. Comprehensive numerical simulations show that the proposed algorithm outperforms existing state-of-the-art DRL algorithms, obtaining a lower error when compared with the optimal global solution (upper boundary) obtained after solving a mathematical programming formulation with perfect forecast information; while strictly enforcing all operational constraints (even in unseen test days).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑