为实现零排放目标,可再生能源发电的增长正使电网控制问题变得越来越具挑战性。近期的 Learning To Run a Power Network(L2RPN)系列竞赛鼓励使用强化学习(RL)来协助人类调度员运行电网。迄今为止提出的所有解决方案都严重限制动作空间,并且基于作用于整个电网的单个智能体,或作用于变电站层面的多个独立智能体。在本工作中,我们提出一种与领域无关的算法,该算法完全基于数据估计状态组件与动作组件之间的相关性。高度相关的状态-动作对会被分组,以创建更简单、可能相互独立的子问题,从而形成不同的学习过程,并降低计算和数据需求。该算法在使用 Grid2op 模拟器获得的电网基准上得到验证,该模拟器已被用于上述各项竞赛;结果表明,我们的算法与领域专家分析相一致。基于这些结果,我们为使用分布式强化学习以改进现有解决方案奠定了具有理论依据的基础。
英文摘要:
The increase of renewable energy generation towards the zero-emission target is making the problem of controlling power grids more and more challenging. The recent series of competitions Learning To Run a Power Network (L2RPN) have encouraged the use of Reinforcement Learning (RL) for the assistance of human dispatchers in operating power grids. All the solutions proposed so far severely restrict the action space and are based on a single agent acting on the entire grid or multiple independent agents acting at the substations level. In this work, we propose a domain-agnostic algorithm that estimates correlations between state and action components entirely based on data. Highly correlated state-action pairs are grouped together to create simpler, possibly independent subproblems that can lead to distinct learning processes with less computational and data requirements. The algorithm is validated on a power grid benchmark obtained with the Grid2Op simulator that has been used throughout the aforementioned competitions, showing that our algorithm is in line with domain-expert analysis. Based on these results, we lay a theoretically-grounded foundation for using distributed reinforcement learning in order to improve the existing solutions.
Learning Topology Actions for Power Grid Control: A Graph-Based Soft-Label Imitation Learning Approach
Fraunhofer Institute for Energy Economics and Energy System Technology (IEE)(弗劳恩霍夫能源经济与能源系统技术研究所);Intelligent Embedded Systems, University of Kassel(卡塞尔大学智能嵌入式系统系);TenneT TSO B.V.(TenneT TSO公司)
Potentially Optimal Joint Actions Recognition for Cooperative Multi-Agent Reinforcement Learning
School of Computer Science and Technology, Tongji University(同济大学计算机科学与技术学院);Stanford University(斯坦福大学);MOE Key Lab of Embedded System and Service Computing, Tongji University, Shanghai, China(同济大学嵌入式系统与服务计算教育部重点实验室,上海,中国);The University of Hong Kong(香港大学);Shanghai Innovation Institute(上海创新研究院)
Real-Time Simulation and Hardware-in-the-Loop Approaches for Integrating Renewable Energy Sources into Smart Grids: Challenges & Actions
University Grenoble Alpes(格勒诺布尔阿尔卑斯大学);Commissariat à l’énergie atomique et aux énergies alternatives(法国原子能和替代能源委员会);Fraunhofer Institute of Wind Energy and Energy System Technology(弗劳恩霍夫风能及能源系统技术研究所);National Technical University of Athens(雅典国立技术大学);Delft University of Technology(代尔夫特理工大学);University of Strathclyde(斯特拉斯克莱德大学);AIT Austrian Institute of Technology(奥地利技术研究院);Danmarks Tekniske Universitet(丹麦技术大学)