arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.04467eess.SYcs.LGcs.SY
收藏

电网中的状态与动作因子分解

State and Action Factorization in Power Grids

Politecnico di Milano(米兰理工大学)

Gianvito Losapio, Davide Beretta, Marco Mussi, Alberto Maria Metelli, Marcello Restelli

AI总结:

本文提出一种数据驱动、与领域无关的状态-动作相关性分解算法,用于构建独立子问题,并在 Grid2op 电网基准上验证其与专家分析一致,为分布式强化学习改进电网调度奠定基础。

摘要:

为实现零排放目标,可再生能源发电的增长正使电网控制问题变得越来越具挑战性。近期的 Learning To Run a Power Network(L2RPN)系列竞赛鼓励使用强化学习(RL)来协助人类调度员运行电网。迄今为止提出的所有解决方案都严重限制动作空间,并且基于作用于整个电网的单个智能体,或作用于变电站层面的多个独立智能体。在本工作中,我们提出一种与领域无关的算法,该算法完全基于数据估计状态组件与动作组件之间的相关性。高度相关的状态-动作对会被分组,以创建更简单、可能相互独立的子问题,从而形成不同的学习过程,并降低计算和数据需求。该算法在使用 Grid2op 模拟器获得的电网基准上得到验证,该模拟器已被用于上述各项竞赛;结果表明,我们的算法与领域专家分析相一致。基于这些结果,我们为使用分布式强化学习以改进现有解决方案奠定了具有理论依据的基础。

英文摘要:

The increase of renewable energy generation towards the zero-emission target is making the problem of controlling power grids more and more challenging. The recent series of competitions Learning To Run a Power Network (L2RPN) have encouraged the use of Reinforcement Learning (RL) for the assistance of human dispatchers in operating power grids. All the solutions proposed so far severely restrict the action space and are based on a single agent acting on the entire grid or multiple independent agents acting at the substations level. In this work, we propose a domain-agnostic algorithm that estimates correlations between state and action components entirely based on data. Highly correlated state-action pairs are grouped together to create simpler, possibly independent subproblems that can lead to distinct learning processes with less computational and data requirements. The algorithm is validated on a power grid benchmark obtained with the Grid2Op simulator that has been used throughout the aforementioned competitions, showing that our algorithm is in line with domain-expert analysis. Based on these results, we lay a theoretically-grounded foundation for using distributed reinforcement learning in order to improve the existing solutions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑