PowerNet：用于可扩展电网控制的多智能体深度强化学习
PowerNet: Multi-agent Deep Reinforcement Learning for Scalable Powergrid Control
摘要：
本文提出了一种高效的多智能体深度强化学习算法，用于电网协同控制。具体来说，我们考虑分布式发电机（DG），这是第一次制定为一个合作的多智能体强化学习（MARL）问题的分布式逆变器为基础的二次电压控制问题。然后，我们提出了一种新的政策MARL算法，PowerNet，其中每个代理（DG）学习的控制策略的基础上（子）全球奖励，但从其邻近的代理的本地状态。由于一个代理的本地控制对远离它的代理的影响有限，我们利用一种新的空间折扣因子来减少远程代理的影响，加快训练过程，提高可扩展性。此外，一个可区分的，基于学习的通信协议，以促进相邻代理之间的合作。此外，为了减轻在策略学习过程中引入的系统不确定性和随机噪声的影响，我们利用动作平滑因子来稳定策略执行。为了便于培训和评估，我们开发了PGSim，一个高效，高保真的电网仿真平台。在两个微电网设置的实验结果表明，开发的PowerNet优于传统的基于模型的控制，以及几个国家的最先进的MARL算法。分散学习方案和高采样效率也使其适用于大规模电网。
英文摘要：
This paper develops an efficient multi-agent deep reinforcement learning algorithm for cooperative controls in powergrids. Specifically, we consider the decentralized inverter-based secondary voltage control problem in distributed generators (DGs), which is first formulated as a cooperative multi-agent reinforcement learning (MARL) problem. We then propose a novel on-policy MARL algorithm, PowerNet, in which each agent (DG) learns a control policy based on (sub-)global reward but local states from its neighboring agents. Motivated by the fact that a local control from one agent has limited impact on agents distant from it, we exploit a novel spatial discount factor to reduce the effect from remote agents, to expedite the training process and improve scalability. Furthermore, a differentiable, learning-based communication protocol is employed to foster the collaborations among neighboring agents. In addition, to mitigate the effects of system uncertainty and random noise introduced during on-policy learning, we utilize an action smoothing factor to stabilize the policy execution. To facilitate training and evaluation, we develop PGSim, an efficient, high-fidelity powergrid simulation platform. Experimental results in two microgrid setups show that the developed PowerNet outperforms a conventional model-based control, as well as several state-of-the-art MARL algorithms. The decentralized learning scheme and high sample efficiency also make it viable to large-scale power grids.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
