基于深度强化学习的风电电力系统网络安全评估
Deep Reinforcement Learning for Cybersecurity Assessment of Wind Integrated Power Systems
摘要：
可再生能源（RES）在电力电力系统（EPS）中的集成正在迅速增加。虽然包括间歇性RES加上通信和传感设备的大规模部署对于完全智能电网非常重要，但它也扩大了网络威胁格局，有效地使电力系统容易受到网络攻击。本文提出了一种网络安全评估方法，旨在评估EPS的网络物理安全。这项工作考虑到RES的间歇性产生，基于微处理器的电子信息和操作技术（IT/OT）设备引入的漏洞，以及应急分析结果。所提出的方法利用深度强化学习（DRL）和为评估EPS中的漏洞而量身定制的适应性通用漏洞评分系统（CVSS）评分，以便基于N-2应急结果来确定最佳攻击转换策略，即，两个系统元件同时失效。基于文献的电网测试案例进行的数值和实时仿真实验验证了工作的有效性。结果表明，所提出的基于深度Q网络（DQN）的方法在找到最佳攻击策略所需的转换数量方面与图搜索方法非常接近，而无需对系统进行全面观察。此外，实验提出了该方法的可扩展性，通过展示的数量的过渡需要找到最佳的攻击过渡政策，在一个大的系统，如波兰2383总线测试系统。结果表明，所提出的方法需要一个数量级的过渡相比，随机过渡政策。
英文摘要：
The integration of renewable energy sources (RES) is rapidly increasing in electric power systems (EPS). While the inclusion of intermittent RES coupled with the wide-scale deployment of communication and sensing devices is important towards a fully smart grid, it has also expanded the cyber-threat landscape, effectively making power systems vulnerable to cyberattacks. This paper proposes a cybersecurity assessment approach designed to assess the cyberphysical security of EPS. The work takes into consideration the intermittent generation of RES, vulnerabilities introduced by microprocessor-based electronic information and operational technology (IT/OT) devices, and contingency analysis results. The proposed approach utilizes deep reinforcement learning (DRL) and an adapted Common Vulnerability Scoring System (CVSS) score tailored to assess vulnerabilities in EPS in order to identify the optimal attack transition policy based on N-2 contingency results, i.e., the simultaneous failure of two system elements. The effectiveness of the work is validated via numerical and real-time simulation experiments performed on literature-based power grid test cases. The results demonstrate how the proposed method based on deep Q-network (DQN) performs closely to a graph-search approach in terms of the number of transitions needed to find the optimal attack policy, without the need for full observation of the system. In addition, the experiments present the method's scalability by showcasing the number of transitions needed to find the optimal attack transition policy in a large system such as the Polish 2383 bus test system. The results exhibit how the proposed approach requires one order of magnitude fewer transitions when compared to a random transition policy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
