利用纵向联邦强化学习增强网络化微电网的网络韧性
Enhancing Cyber Resilience of Networked Microgrids using Vertical Federated Reinforcement Learning
AI总结：
本文针对网络化微电网的网络韧性提升问题，提出纵向联邦软演员-评论家（FedSAC）算法，在保护多方数据隐私的同时有效捕捉微电网间耦合动态，并通过IEEE 123节点基准系统验证了其抵御对抗攻击的有效性。
摘要：
本文提出了一种新颖的联邦强化学习（Fed-RL）方法，以增强网络化微电网的网络韧性。我们构建了一个具有韧性的强化学习（RL）训练设置，该设置（a）生成在构网型（GFM）逆变器的一次控制参考信号处注入对抗性动作的回合式轨迹，并（b）训练RL智能体（或控制器）以减轻所注入对抗行为的影响。为规避多方共有的网络化电网中的数据共享问题及专有隐私顾虑，我们引入了联邦机器学习的理念，并提出一种新颖的Fed-RL算法来训练RL智能体。为此，传统采用解耦独立环境的横向Fed-RL方法无法捕捉网络化微电网中的耦合动态，这促使我们提出一种多智能体纵向联邦的演员-评论家算法变体，即联邦软演员-评论家（FedSAC）算法。我们创建了一个定制化仿真设置，在与OpenAI Gym接口兼容的GridLAB-D/HELICS联合仿真平台中封装微电网动态，用于训练RL智能体。最后，通过由三个耦合微电网组成的修改版IEEE 123节点基准测试系统的数值算例验证了所提方法。
英文摘要：
This paper presents a novel federated reinforcement learning (Fed-RL) methodology to enhance the cyber resiliency of networked microgrids. We formulate a resilient reinforcement learning (RL) training setup which (a) generates episodic trajectories injecting adversarial actions at primary control reference signals of the grid forming (GFM) inverters and (b) trains the RL agents (or controllers) to alleviate the impact of the injected adversaries. To circumvent data-sharing issues and concerns for proprietary privacy in multi-party-owned networked grids, we bring in the aspects of federated machine learning and propose a novel Fed-RL algorithm to train the RL agents. To this end, the conventional horizontal Fed-RL approaches using decoupled independent environments fail to capture the coupled dynamics in a networked microgrid, which leads us to propose a multi-agent vertically federated variation of actor-critic algorithms, namely federated soft actor-critic (FedSAC) algorithm. We created a customized simulation setup encapsulating microgrid dynamics in the GridLAB-D/HELICS co-simulation platform compatible with the OpenAI Gym interface for training RL agents. Finally, the proposed methodology is validated with numerical examples of modified IEEE 123-bus benchmark test systems consisting of three coupled microgrids.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
