Model-based Vol/VAR optimization method is widely used to eliminate voltage violations and reduce network losses. However, the parameters of active distribution networks(ADNs) are not onsite identified, so significant errors may be involved in the model and make the model-based method infeasible. To cope with this critical issue, we propose a novel two-stage deep reinforcement learning (DRL) method to improve the voltage profile by regulating inverter-based energy resources, which consists of offline stage and online stage. In the offline stage, a highly efficient adversarial reinforcement learning algorithm is developed to train an offline agent robust to the model mismatch. In the sequential online stage, we transfer the offline agent safely as the online agent to perform continuous learning and controlling online with significantly improved safety and efficiency. Numerical simulations on IEEE test cases not only demonstrate that the proposed adversarial reinforcement learning algorithm outperforms the state-of-art algorithm, but also show that our proposed two-stage method achieves much better performance than the existing DRL based methods in the online application.
University of California San Diego（加州大学圣地亚哥分校）；NEC Laboratories America（NEC美国实验室）
Increasing adoption of solar photovoltaic (PV) presents new challenges to modern power grid due to its variable and intermittent nature. Fluctuating outputs from PV generation can cause the grid violating voltage operation limits. PV smart inverters (SIs) provide a fast-response method to regulate voltage by modulating real and/or reactive power at the connection point. Yet existing local autonomous control scheme of SIs is based on local information without coordination, which can lead to suboptimal performance. In this paper, a deep reinforcement learning (DRL) based algorithm is developed a…
The distributed Volt/Var control (VVC) methods have been widely studied for active distribution networks(ADNs), which is based on perfect model and real-time P2P communication. However, the model is always incomplete with significant parameter errors and such P2P communication system is hard to maintain. In this paper, we propose an online multi-agent reinforcement learning and decentralized control framework (OLDC) for VVC. In this framework, the VVC problem is formulated as a constrained Markov game and we propose a novel multi-agent constrained soft actor-critic (MACSAC) reinforcement learn…
Tsinghua-Berkeley Shenzhen Institute, Tsinghua University（清华大学清华-伯克利深圳学院）；Shanghai University of Electric Power（上海电力大学）；Tsinghua University（清华大学）；Beihang University（北京航空航天大学）
Reference-model-assisted deep reinforcement learning (DRL) for inverter-based Volt-Var Control (IB-VVC) in active distribution networks is proposed. We investigate that a large action space increases the learning difficulties of DRL and degrades the optimization performance in the process of generating data and training neural networks. To reduce the action space of DRL, we design a reference-model-assisted DRL approach. We introduce definitions of the reference model, reference-model-based optimization, and reference actions. The reference-model-assisted DRL learns the residual actions betwee…
This paper focuses on the effective use of smart inverters for Volt/Var control (VVC) on a distribution system. New smart inverters offer Var support capability but for their effective use they need to be coordinated with existing Volt/Var schemes. A new VVC scheme is proposed to facilitate such coordination. The proposed scheme decomposes the problem into two levels. The first level uses Load Tap Changer (LTC) and Voltage Regulators (VRs) and coordinates their control with smart inverters to adjust the voltage level on the circuit to keep the voltages along the circuit within the desired rang…
Recent research on multilevel inverters shows exciting properties, including the potential to generate multiple output voltages and integrated voltage boosting. However, most presented inverter topologies have a restricted number of output voltage levels and limited output voltage boosting ratio. In addition, balancing the voltage of capacitors in multilevel converters is very important and should be considered in the topology or control method. This paper describes a generalized switched-capacitor circuit topology for multilevel dc-to-ac inverters that can be developed for the desired output …
The high R/X ratio of typical distribution systems makes the system voltage vulnerable to the active power injection from distributed energy resources (DERs). Moreover, the intermittent and uncertain nature of the DER generation brings new challenges to the voltage control. This paper proposes a two-stage stochastic optimization strategy to optimally place the PV smart inverters with Volt-VAr capability for distribution systems with high photovoltaic (PV) penetration to mitigate voltage violation issues. The proposed optimization strategy enables a planningstage guide for upgrading the existin…
Fast-acting smart inverters that utilize preset operating conditions to determine real and reactive power injection/consumption can create voltage instabilities (over-voltage, voltage oscillations and more) in an electrical distribution network if set-points are not properly configured. In this work, linear distribution power flow equations and droop-based Volt-Var and Volt-Watt control curves are used to analytically derive a stability criterion using \lyapnouv analysis that includes the network operating condition. The methodology is generally applicable for control curves that can be repres…
Advanced microgrids consisting of distributed energy resources interfaced with multi-inverter systems are becoming more common. Consequently, the effectiveness of voltage and frequency regulation in microgrids using conventional droop-based methodologies is challenged by uncertainty in the sizeand schedule of loads. This article proposes an isochronous architecture of parallel inverters with only voltage-active power droop (VP-D) control for improving active power sharing as well as plug-and-play of multi-inverter based distributed energyresources (DERs). In spite of not employing explicit con…
This paper shows that a relation can be found between the voltage at the terminals of an inverter-interfaced Renewable Energy Source RES and its optimal reactive power support. This relationship, known as Volt-Var Curve VVC, enables the decentral operation of RES for Active Voltage Management (AVM). In this paper, the decentralized AVM technique is modified to consider the effects of the realistic operational constraints of RES. The AVM technique capitalizes on the reactive power support capabilities of inverters to achieve the desired objective in unbalanced active Low-Voltage Distribution Sy…
Optimal Volt/VAR control (VVC) in distribution networks relies on an effective coordination between the conventional utility-owned mechanical devices and the smart residential photovoltaic (PV) inverters. Typically, a central controller carries out a periodic optimization and sends setpoints to the local controller of each device. However, instead of tracking centrally dispatched setpoints, smart PV inverters can cooperate on a much faster timescale to reach optimality within a PV inverter group. To accommodate such PV inverter groups in the VVC architecture, this paper proposes a bi-level opt…
Design and implementation of an optimal and robust single-loop voltage controller is proposed for single-phase grid forming voltage source inverter (VSI). The objective of the proposed controller is to have good reference tracking and disturbance rejection. Uncertain nature of loads can significantly alter system's behavior, especially during heavily loaded conditions, and impose an uncertainty in the dynamic model. This deteriorates the robustness of the controller severely. H-Infinity-based controller design is proposed in this article to address this issue. The required objectives of the op…
Multilevel inverters (MLIs) are popular because of their advantages such as improved output voltage quality, lower switching losses, low EMI, and ability to handle higher voltage and power levels. To generate the desired output voltage in the MLIs, there are different switching methods including multicarrier pulse-width modulation (PWM), selective harmonic elimination (SHE), and multilevel space-vector PWM (SVPWM) methods. The SHE method minimizes the number of switching and hence the switching losses. Also, the method eliminates or attenuates specified low-order harmonics and adjusts the fund…
This paper proposes a new DC-AC converter with a less number of semiconductor switches and therefore gate drive and control circuit components. A salient feature of this inverter is its pure sinusoidal line-to-line voltages with no need for output filter. Also, the proposed inverter improves the voltage utilization factor of the input dc supply compared to four-switch three-phase inverter (first best topology in the literature from the number of switches point of view) and standard six-switch inverter with sine PWM. The proposed inverter is capable of operating with a wide range of output volt…
Autonomous droop control PV inverters have improved voltage regulation compared to the inverters without grid support functions, but more flexible control techniques will be required as the number of solar photovoltaic (PV) installations increases. This paper studies three inverter future deployment scenarios with droop control inverters, non-exporting inverters, and coordinated inverter control (CIC). The network operation and the interaction between various inverter control methods are studied by simulating inverter operation on two low-voltage networks. Considering 30% PV penetration as the…
In a purely inverter-based microgrid, both grid-forming (GFM) and grid-following (GFL) inverters will have a crucial role to play in frequency/voltage regulation and maintaining power sharing through their grid support capabilities. Consequently, the coordination between these two technologies becomes increasingly important for optimal system performance. However, the existing work does not consider GFL's potential to participate in a secondary control in coordination with GFM, thus not able to utilize the full capability of inverter resources. In this paper, we show that it is possible to ful…
This paper investigates a multilevel inverter with a capability that produces a wide voltage range with high quality. The selective harmonic elimination (SHE) method is considered for a single-phase 5-level cascaded H-bridge (CHB) inverter, in which the particle swarm optimization (PSO) algorithm solves the nonlinear equations. However, eliminating the low-order harmonics has been challenging when a low range of output voltage is required. To surmount such challenges and access to a wide range of output voltage, an adjustable dc-link is introduced that allows the inverter to increase the modul…
Photovoltaic (PV) smart inverters can improve the voltage profile of distribution networks. A multi-objective optimization framework for coordination of reactive power injection of smart inverters and tap operations of on-load tap changers (OLTCs) for multi-phase unbalanced distribution systems is proposed. The optimization objective is to minimize voltage deviations and the number of tap operations simultaneously. A novel linearization method is proposed to linearize power flow equations and to convexify the problem, which guarantees convergence of the optimization and less computation costs.…
We consider the problem of stability analysis for distribution grids with droop-controlled inverters and dynamic distribution power lines. The inverters are modeled as voltage sources with controllable frequency and amplitude. This problem is very challenging for large networks as numerical simulations and detailed eigenvalue analysis are impactical. Motivated by the above limitations, we present in this paper a systematic and computationally efficient framework for stability analysis of inverter-based distribution grids. To design our framework, we use tools from singular perturbation and Lya…
As more distributed energy resources (DERs) are connected to the power grid, it becomes increasingly important to ensure safe and effective coordination between legacy voltage regulation devices and inverter-based DERs. In this work, we show how a distribution circuit model, composed of two LTCs and two inverter devices, can create voltage oscillations even with reasonable choices of control parameters. By modeling the four-device circuit as a switched affine hybrid system, we analyze the system's oscillatory behavior, both during normal operation and after a cyber-physical attack. Through the…
This paper proposes a new distributed consensus-based control method for voltage and frequency control of fully inverter-based islanded microgrids (MGs). The proposed method includes the active power sharing in voltage control to improve the reactive power sharing accuracy and thus generalizes some existing secondary frequency and voltage control methods. Firstly, frequency is regulated by distributed secondary frequency control. Secondly, voltage is regulated by distributed average voltage control and decentralized individual voltage control. It offers a tunable trade-off between voltage regu…
Unbalanced and nonlinear loads connected to microgrids (MG) with local distributed energy resources (DERs) are two of the leading causes of power quality problems. Nonlinearloads introduce voltage and current harmonics, and single-phase loads can cause voltage and current imbalances in a three-phase network. This paper presents a hierarchical control scheme for voltage-controlled photovoltaic (PV) inverters with unbalanced and nonlinear loads in micro-grids. The hierarchical control consists of primary control and voltage compensation control(VCC) and a DC voltage regulator (VR). The primary c…
This paper presents a two-stage zonal Volt/VAR control scheme for coordinating inverter-based resources (IBR) with utility-owned voltage regulators (VR) to regulate voltage in unbalanced 3-phase distribution systems. First, correlations between nodal voltages are derived from nodal voltage sensitivity studies. Then, the feeder is partitioned into non-overlapping, weakly-coupled voltage control zones based on nodal voltage correlations. IBR are used in the first stage to regulate voltage changes continuously and VR are used in the second stage to regulate large voltage deviations. An online VR …
Regression-based Inverter Control for Decentralized Optimal Power Flow and Voltage Regulation
2 Department of Electrical Engineering \& Computer Sciences, UC Berkeley, Berkeley, USA；3 Department of Mechanical Engineering, UC Berkeley, Berkeley, USA；4 Delft Center for Systems；Control, Delft University of Technology, Delft, The Netherlands
This paper presents current control method for a grid-connected partial resonant soft switching inverter. This inverter does not use an electrolytic capacitor and is capable of boosting and bucking the voltage. Grid-connected inverters are used to integrate distributed energy sources to the grid. Current control is vital in meeting the standards and requirements when connecting to the grid. The closed-loop current regulation for this type of converters is analyzed and design guidelines are provided. The control is implemented in the synchronous frame. In addition active damping techniques usin…
Inverter-based resources (IBRs) are required to provide dynamic voltage support (DVS) during voltage dips to enhance the low-voltage ride-through capability. In this paper, we develop a model-free control method to achieve the optimal DVS (ODVS) without relying on the knowledge of grid parameters. Delving into the optimum trajectory of the ODVS problem, it is found that either the current constraint and the maximum active power constraint of IBRs are binding or one of the constraints is binding. This inspires us to search for the optimum in a closed-loop way by a perturb-and-observe (P&O)-base…
This paper presents a method for controlling the voltage of inverter-based Microgrids by proposing a new scale-free distributed cooperative controller. The communication network is modeled by a general time-varying graph which enhances the resilience of the proposed protocol against communication link failure, data packet loss, and fast plug and play operation in the presence of arbitrarily communication delays. The proposed scale-free distributed cooperative controller is independent of any information about the communication system and the size of the network (i.e., the number of distributed…
This paper presents a novel grid-forming voltage control strategy for a battery energy storage system to maintain balanced three-phase output voltages when serving unbalanced loads. A stationary reference frame ($αβ$) based control scheme is proposed to regulate positive-sequence and negative-sequence voltages. Compared with the conventional rotating reference frame ($dq$) based control scheme, the proposed scheme shows better dynamic performance. Then, we analyze the system zero-sequence network and propose adding a grounding transformer to the Y/Yg connected output transformer to further red…
To mitigate issues related to the growth of variable smart loads and distributed generation, distribution system operators (DSO) now make it binding for prosumers with inverters to operate under pre-set rules. In particular, the maximum active and reactive power set points for prosumers are based on local voltage measurements to ensure that inverter output does not cause voltage violations. However, such actions, as observed in this work, restrict the range available for local energy management, with more adverse losses on arbitrage profits for prosumers located farther away from the substatio…
This paper employs a supervised machine learning (ML) algorithm to propose an integrated fault detection and diagnosis (FDD) and fault-tolerant control (FTC) strategy to detect, diagnose, and classify the grid faults and correct the input voltage before affecting the grid-connected distributed energy resources (DER) inverters. This controller can mitigate the impact of grid faults on inverters by predicting and modifying the time series of their input voltage. Simulation results show the effectiveness of the proposed controller and evaluate its operating performance.
Grid-forming inverter-based autonomous microgrids present new operational challenges as the stabilizing rotational inertia of synchronous machines is absent. The design of efficient control policies for grid-forming inverters is, however, a non-trivial problem where multiple performance objectives need to be satisfied, including voltage/frequency regulation, current limiting capabilities, as well as active power sharing and a scalable operation. We propose in this paper a novel control architecture for frequency and voltage control which allows current limitation via an inner loop, active powe…
This article proposes the design of H-infinity-based robust current controller for single-phase grid-feeding voltage source inverter with an LCL filter. The main objective of the proposed controller is to have good reference tracking, disturbance rejection and sufficient LCL resonance damping under a large range of variations of grid impedance. Based on the aforementioned performance requirements, frequency dependent weighting functions are designed. Subsequently, the sub-optimal control problem is formulated and solved to determine the stabilizing controller. Computational footprint of the co…
The lower order harmonic elimination using space vector pulse width modulation (SVPWM) technique is possible by introducing the dwell time division coefficient 'k' in space vector. In this paper, torque ripple analysis is carried out for space vector based harmonic elimination (SVHE) scheme and the obtained results are compared with the existing schemes such as conventional and advanced bus clamping SVPWM schemes. Further, the obtained results are validated through simulation results. The SVHE scheme gives the lower torque ripple than the conventional and advance bus clamping schemes at higher…
This paper investigates the dynamic voltage support (DVS) control of inverter-based resources (IBRs) under voltage sags to enhance the low-voltage ride-through performance. We first revisit the prevalent droop control from an optimization perspective to elaborate on why it usually suffers from suboptimality. Then, we formulate the DVS problem as an optimization program that maximizes the positive-sequence voltage magnitude at the point of common coupling (PCC) subject to the current, active power, and stability constraints. The program is inherently nonconvex due to the active power limits, of…
