High-level penetration of intermittent renewable energy sources (RESs) has introduced significant uncertainties into modern power systems. In order to rapidly and economically respond to the fluctuations of power system operating state, this paper proposes a safe deep reinforcement learning (SDRL) based method for fast solution of real-time optimal power flow (RT-OPF) problems. The proposed method considers the volatility of RESs and temporal constraints, and formulates the RT-OPF as a Constrained Markov Decision Process (CMDP). In the training process, the proposed method hybridizes the proximal policy optimization (PPO) and the primal-dual method. Instead of integrating the constraint violation penalty with the reward function, its actor gradients are estimated by a Lagrange advantage function which is derived from two critic systems based on economic reward and violation cost. The decoupling of reward and cost alleviates reward sparsity while improving critic approximation accuracy. Moreover, the introduction of Lagrange multipliers enables the agent to comprehend the trade-off between optimality and feasibility. Numerical tests are carried out and compared with penalty-based DRL methods on the IEEE 9-bus, 30-bus, and 118-bus test systems. The results show that the well-trained SDRL agent can significantly improve the computation efficiency while satisfying the security constraints and optimality requirements.
SAVER: Safe Learning-Based Controller for Real-Time Voltage Regulation
快速和安全的电压调节算法可以作为实现现代配电网中高水平可再生能源渗透的基本方案。面对不确定甚至未知的配电网模型和快速变化的功率注入，已经提出了无模型深度强化学习（DRL）算法来找到逆变器的无功功率注入，同时优化电压曲线。然而，这种数据驱动的控制器不能保证满足硬操作约束，例如将电压分布保持在标称值的一定范围内。为此，我们提出了SAVER：SAfe VoltagE Regulator，它由RL学习器和专门设计的计算高效的安全投影层组成。SAVER为一组DRL算法提供即插即用接口，保证系统电压在安全范围内。对真实数据的数值仿真验证了该算法的性能。
Safe and Reliable Training of Learning-Based Aerospace Controllers
Stanford University（斯坦福大学）；The Hebrew University of Jerusalem（耶路撒冷希伯来大学）；IT University of Copenhagen（哥本哈根 IT 大学）；Google（谷歌）；GE Aerospace Research（GE航空航天研究院）；Center for AI Safety（人工智能安全中心）；Air Force Research Laboratory（空军研究实验室）
Safe Model-based Off-policy Reinforcement Learning for Eco-Driving in Connected and Automated Hybrid Electric Vehicles
Center for Automotive Research, The Ohio State University（俄亥俄州立大学汽车研究中心）；Bending Spoons（本丁斯普恩斯）；Department of Electrical and Computer Engineering, The Ohio State University（俄亥俄州立大学电气与计算机工程系）
Fast State Stabilization using Deep Reinforcement Learning for Measurement-based Quantum Feedback Control
University of New South Wales（新南威尔士大学）；University of Newcastle（纽卡斯尔大学）；Australian National University（澳大利亚国立大学）；RIKEN Center for Quantum Computing（理化学研究所量子计算中心）
Iterative Learning Control-Informed Reinforcement Learning for Batch Process Control
State Key Laboratory of Industrial Control Technology, Institute of Cyber-Systems and Control, Zhejiang University（工业控制技术国家重点实验室，系统与控制研究所，浙江大学）；Department of Chemical Engineering, Chung-Yuan Christian University（Chung-Yuan Christian University 化学工程系）
Facilitating Reinforcement Learning for Process Control Using Transfer Learning: Overview and Perspectives
College of Control Science and Engineering Zhejiang University（控制科学与工程学院浙江大学）；Department of Chemical Engineering Chung-Yuan Christian University（化学工程系 Chung-Yuan Christian University）
The Pump Scheduling Problem: A Real-World Scenario for Reinforcement Learning
Normandie Université（诺曼底大学）；INSA Rouen（里昂大学鲁昂分校）；LITIS Rouen（鲁昂LITIS研究所）；Technical University of Kaiserslautern（凯撒斯劳滕技术大学）；SAM Kaiserlaustern（凯撒斯劳滕SAM）
Representation Learning Enhanced Deep Reinforcement Learning for Optimal Operation of Hydrogen-based Multi-Energy Systems
School of Automation Science and Engineering, Xi’an Jiaotong University（自动化科学与工程学院，西安交通大学）；Center for Intelligent and Networked Systems, Department of Automation, BNRist, Tsinghua University（智能与网络化系统中心，自动化系，清华大学）；Department of Electrical Engineering and Computer Sciences, University of California（电气工程与计算机科学系，加州大学）
