arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.00304eess.SYcs.SY
收藏

现代电力系统安全强化学习方法综述

A Review of Safe Reinforcement Learning Methods for Modern Power Systems

Tong Su, Tong Wu, Junbo Zhao, Anna Scaglione, Le Xie

AI总结:

本文综述了安全强化学习在现代电力系统运行与控制中的应用,指出其通过Lagrangian松弛、安全层和Lyapunov函数等主动机制严格保障安全约束,并总结了现有技术、部署场景、真实案例及未来挑战。

摘要:

鉴于现代电力系统中更全面测量数据的可用性,强化学习(RL)在运行和控制方面引起了广泛关注。传统RL依赖于与环境的试错交互和奖励反馈,这通常会导致探索不安全的运行区域并执行不安全的动作,尤其是在部署于真实世界的电力系统中时。为了应对这些挑战,人们提出了安全RL,以在确保满足安全约束的同时优化运行目标,在训练和部署的全过程中将动作和状态保持在安全区域内。传统RL通常仅依赖人为设计的不安全动作惩罚项,而本文所综述的安全RL方法主要利用先进的主动机制。这些机制包括Lagrangian松弛、安全层等技术,以及像Lyapunov函数这样的理论保证,以严格强制执行安全边界。本文全面综述了安全RL方法及其在各个电力系统运行和控制领域的应用,包括安全控制、实时运行、运行规划以及新兴领域。文章总结了现有的安全RL技术,评估了其性能,分析了适用的部署场景,并考察了算法基准和应用环境。本文还重点介绍了真实世界的实施案例,并指出了关键挑战,如大规模系统中的可扩展性和不确定性下的鲁棒性,为推进安全RL在现代电力系统中的可靠集成与部署提供了潜在解决方案并规划了未来方向。

英文摘要:

Given the availability of more comprehensive measurement data in modern power systems, reinforcement learning (RL) has gained significant interest in operation and control. Conventional RL relies on trial-and-error interactions with the environment and reward feedback, which often leads to exploring unsafe operating regions and executing unsafe actions, especially when deployed in real-world power systems. To address these challenges, safe RL has been proposed to optimize operational objectives while ensuring safety constraints are met, keeping actions and states within safe regions throughout both training and deployment. Rather than relying solely on manually designed penalty terms for unsafe actions, as is common in conventional RL, safe RL methods reviewed here primarily leverage advanced and proactive mechanisms. These include techniques such as Lagrangian relaxation, safety layers, and theoretical guarantees like Lyapunov functions to rigorously enforce safety boundaries. This paper provides a comprehensive review of safe RL methods and their applications across various power system operations and control domains, including security control, real-time operation, operational planning, and emerging areas. It summarizes existing safe RL techniques, evaluates their performance, analyzes suitable deployment scenarios, and examines algorithm benchmarks and application environments. The paper also highlights real-world implementation cases and identifies critical challenges such as scalability in large-scale systems and robustness under uncertainty, providing potential solutions and outlining future directions to advance the reliable integration and deployment of safe RL in modern power systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑