Accurate knowledge of the distribution system topology and parameters is required to achieve good voltage controls, but this is difficult to obtain in practice. This paper develops a model-free approach based on the surrogate model and deep reinforcement learning (DRL). We have also extended it to deal with unbalanced three-phase scenarios. The key idea is to learn a surrogate model to capture the relationship between the power injections and voltage fluctuation of each node from historical data instead of using the original inaccurate model affected by errors and uncertainties. This allows us to integrate the DRL with the learned surrogate model. In particular, DRL is applied to learn the optimal control strategy from the experiences obtained by continuous interactions with the surrogate model. The integrated framework contains training three networks, i.e., surrogate model, actor, and critic networks, which fully leverage the strong nonlinear fitting ability of deep learning and DRL for online decision making. Several single-phase approaches have also been extended to deal with three-phase unbalance scenarios and the simulation results on the IEEE 123-bus system show that our proposed method can achieve similar performance as those that use accurate physical models.
Photovoltaic (PV) smart inverters can improve the voltage profile of distribution networks. A multi-objective optimization framework for coordination of reactive power injection of smart inverters and tap operations of on-load tap changers (OLTCs) for multi-phase unbalanced distribution systems is proposed. The optimization objective is to minimize voltage deviations and the number of tap operations simultaneously. A novel linearization method is proposed to linearize power flow equations and to convexify the problem, which guarantees convergence of the optimization and less computation costs.…
Autonomous Pressure Control in MuVacAS via Deep Reinforcement Learning and Deep Learning Surrogate Models
HI Iberia（HI伊比利亚）；Department of Mathematics, Universidad Carlos III de Madrid（数学系，卡洛斯三世大学）；Gregorio Millán Barbany Institute, Universidad Carlos III de Madrid（格里戈里·米兰·巴尔巴尼研究所，卡洛斯三世大学）；IFMIF-DONES Spain（IFMIF-DONES西班牙）
This paper develops a model-free volt-VAR optimization (VVO) algorithm via multi-agent deep reinforcement learning (MADRL) in unbalanced distribution systems. This method is novel since we cast the VVO problem in unbalanced distribution networks to an intelligent deep Q-network (DQN) framework, which avoids solving a specific optimization model directly when facing time-varying operating conditions of the systems. We consider statuses/ratios of switchable capacitors, voltage regulators, and smart inverters installed at distributed generators as the action variables of the DQN agents. A delicat…
This paper proposes a novel two-layer Volt/VAR control (VVC) framework to regulate the voltage profiles across an unbalanced active distribution system, which achieves both the efficient open-loop optimization and accurate closed-loop tracking. In the upper layer, the conventional voltage regulation devices with discrete and slow-response characteristics are optimally scheduled to regulate voltage profiles in an hourly timescale while improving economic operations based on the receding horizon optimization (RHO) in a centralized manner. A generalized linearized branch flow model (G-LBFM) is de…
In this paper, we introduce a new framework to address the problem of voltage regulation in unbalanced distribution grids with deep photovoltaic penetration. In this framework, both real and reactive power setpoints are explicitly controlled at each solar panel smart inverter, and the objective is to simultaneously minimize system-wide voltage deviation and maximize solar power output. We formulate the problem as a Markov decision process with continuous action spaces and use proximal policy optimization, a reinforcement learning-based approach, to solve it, without the need for any forecast o…
This paper presents a zonal Volt/VAR control scheme that coordinates Photovoltaic (PV) inverters for providing voltage regulation on 3-phase unbalanced distribution feeders. Voltage sensitivity studies are conducted to uncover the dependency between nodal voltage changes and the reactive power injections at nodes with smart PV inverters. A fast incremental clustering method is developed to divide the distribution feeder into weakly coupled zones based on correlations of nodal voltage sensitivities. Because each zone is weakly coupled, voltage of each zone can be controlled independently. Thus,…
As a consequence of the transition to distributed and renewable energy systems, some distribution system operators are increasingly concerned about power quality, including steady-state voltage volatility. In this paper, we study the control of real and reactive power injections by inverter-based distributed energy resources to regulate voltage magnitudes and phase angles measured by sensors positioned across 3-phase unbalanced distribution grids. Our proposed controllers are agnostic to the location and type of communications supporting energy resource operations. To design the controllers, w…
For active distribution networks (ADNs) integrated with massive inverter-based energy resources, it is impractical to maintain the accurate model and deploy measurements at all nodes due to the large-scale of ADNs. Thus, current models of ADNs are usually involving significant errors or even unknown. Moreover, ADNs are usually partially observable since only a few measurements are available at pilot nodes or nodes with significant users. To provide a practical Volt/Var control (VVC) strategy for such networks, a data-driven VVC method is proposed in this paper. Firstly, the system response pol…
This paper proposes a tractable distributionally robust chance-constrained conservation voltage reduction (DRCC-CVR) method with enriched data-based ambiguity set in unbalanced three-phase distribution systems. The increasing penetration of distributed renewable energy not only brings clean power but also challenges the voltage regulation and energy-saving performance of CVR by introducing high uncertainties to distribution systems. In most cases, the conventional robust optimization methods for CVR only provide conservative solutions. To better consider the impacts of load and PV generation u…
A Digital Twin Framework for Generation-IV Reactors with Reinforcement Learning-Enabled Health-Aware Supervisory Control
organization= Department of Aerospace Engineering, University of Michigan , addressline= 1320 Beal Ave , city= Ann Arbor , postcode= 48109 , state= MI , country= USA；Engineering Division, U.S. DOE Idaho National Laboratory , addressline= 1955 N. Fremont Ave. , city= Idaho Falls , postcode= 83415 , state= ID , country= USA；organization= Michigan Institute for Computational Discovery \& Engineering (MICDE), University of Michigan , addressline= 915 E Washington St, Suite 3530 , city= Ann Arbor , postcode= 48109 , state= MI , country= USA
