The uncertainty of distributed renewable energy brings significant challenges to economic operation of microgrids. Conventional online optimization approaches require a forecast model. However, accurately forecasting the renewable power generations is still a tough task. To achieve online scheduling of a residential microgrid (RM) that does not need a forecast model to predict the future PV/wind and load power sequences, this paper investigates the usage of reinforcement learning (RL) approach to tackle this challenge. Specifically, based on the recent development of Model-Based Reinforcement Learning, MuZero, we investigate its application to the RM scheduling problem. To accommodate the characteristics of the RM scheduling application, a optimization framework that combines the modelbased RL agent with the mathematical optimization technique is designed, and long short-term memory (LSTM) units are adopted to extract features from the past renewable generation and load sequences. At each time step, the optimal decision is obtained by conducting Monte-Carlo tree search (MCTS) with a learned model and solving an optimal power flow sub-problem. In this way, this approach can sequentially make operational decisions online without relying on a forecast model. The numerical simulation results demonstrate the effectiveness of the proposed algorithm.
Lately, increasing number of electric vehicles (EVs) in residential parking station has become an important issue, because excessive number of EVs can destabilize the power system during peak hours with high charging power requested. When the power system of the residential parking station takes the structure of microgrid (MG), power provision for the EVs requires efficient power management scheme. To minimize the maintenance cost of the MG and maintain the grid stability, the MG needs to balance the charging/discharging power of EVs in the parking station. To achieve these goals, this paper p…
Risk analysis is currently not quantified in microgrid resource scheduling optimization. This paper conducts a conditional value at risk (cVaR) analysis on a grid-disconnected residential microgrid with distributed energy resources (DER). We assume the infrastructure to set up an ad-hoc microgrid is already in place for a residential neighborhood with power sources such as photovoltaic (PV), diesel, and battery energy storage system (BESS). With this scenario in mind, we solve day-ahead scheduling to optimally allocate various resources to match demand in scenarios where neighborhoods, especia…
Microgrids with AC/DC architecture benefit from advantages of both AC and DC power. In this paper, daily operation problem for a zero-carbon AC/DC microgrid in presence of electric vehicles (EVs) is considered. In this framework, EVs' batteries are mobile energy storage systems, which allow desirable operation of the microgrid during peak demand hours. This study shows in absence of storage system, EVs' batteries can be properly managed to satisfy the system requirements. In the case studies, several sensitivity analyses based on variations in battery degradation costs, solar irradiance, and i…
A cooperative energy scheduling method is proposed that allows joint energy optimization for a group of microgrids to achieve cost savings that the microgrids could not achieve individually. The discussed microgrids may be commercial entities in a distribution network under utility electricity rate plans comprising both Time of Use (ToU) and peak demand charge. Defining a stable operation as a situation where all microgrids would be willing to participate, it is shown that under such rate plans and in particular due to the peak demand charge, a cost distribution that is seemingly fair does not…
This letter investigates a Branching Dueling Q-Network (BDQ) based online operation strategy for a microgrid with distributed battery energy storage systems (BESSs) operating under uncertainties. The developed deep reinforcement learning (DRL) based microgrid online optimization strategy can achieve a linear increase in the number of neural network outputs with the number of distributed BESSs, which overcomes the curse of dimensionality caused by the charge and discharge decisions of multiple BESSs. Numerical simulations validate the effectiveness of the proposed method.
In order to prevent the potential frequency instability due to the high Power Electronics (PE) penetration under an unintentional islanding event, this paper presents a novel microgrid scheduling model which explicitly models the system frequency dynamics as well as the long/short term uncertainty associated with renewable energy resources and load. Synthetic Inertia (SI) control is applied to regulating the active power output of the Inverter-Based Generators (IBGs) to support the post-islanding frequency evaluation. The uncertainty associated with the noncritical load shedding is explicitly …
The development of renewable energy generation empowers microgrids to generate electricity to supply itself and to trade the surplus on energy markets. To minimize the overall cost, a microgrid must determine how to schedule its energy resources and electrical loads and how to trade with others. The control decisions are influenced by various factors, such as energy storage, renewable energy yield, electrical load, and competition from other microgrids. Making the optimal control decision is challenging, due to the complexity of the interconnected microgrids, the uncertainty of renewable energ…
With the popularization of the electric vehicles (EVs), EV charging demand is becoming an important load in the building. Considering the mobility of EVs from building to building and their uncertain charging demand, it is of great practical interest to control the EV charging process in a microgrid of buildings to optimize the total operation cost while ensuring the transmission safety between the microgrid and the main grid. We consider this important problem in this paper and make the following contributions. First, we formulate this problem as a Markov decision process to capture the uncer…
This paper presents an optimization approach based on the mixed-integer programming (MIP) to maximize the profit of the Microgrid (MG) while minimizing the risk in profit (RIP) in the presence of demand response program (DRP). RIP is defined as the risk of gaining less profit from the desired profit values. The uncertainties associated with the RESs and loads are modeled using normal, Beta, and Weibull distribution functions. The simulation studies are performed in GAMS and MATLAB for 5 random days in a year. The simulation results show that RIP is reduced when downside risk constraint (DRC) i…
Online decision-making in the presence of uncertain future information is abundant in many problem domains. In the critical problem of energy generation scheduling for microgrids, one needs to decide when to switch energy supply between a cheaper local generator with startup cost and the costlier on-demand external grid, considering intermittent renewable generation and fluctuating demands. Without knowledge of future input, competitive online algorithms are appealing as they provide optimality guarantees against the optimal offline solution. In practice, however, future input, e.g., wind gene…
Islamic Azad University（伊斯兰阿扎德大学）；Iranian University of Science and Technology（伊朗科技大学）；Shiraz University of Technology（设拉子理工大学）；Electric reliability council of Texas（得克萨斯州电力可靠性委员会）
In terms of the operation of microgrids, optimal scheduling is a vital issue that must be taken into account. In this regard, this paper proposes an effective framework for optimal scheduling of renewable microgrids considering energy storage devices, wind turbines, micro turbines. Due to the nonlinearity and complexity of operation problems in microgrids, it is vital to use an accurate and robust optimization technique to efficiently solve this problem. To this end, in the proposed framework, the teacher learning-based optimization is utilized to efficiently solve the scheduling problem in th…
The emerging paradigm of interconnected microgrids advocates energy trading or sharing among multiple microgrids. It helps make full use of the temporal availability of energy and diversity in operational costs when meeting various energy loads. However, energy trading might not completely absorb excess renewable energy. A multi-energy management framework including fuel cell vehicles, energy storage, combined heat and power system, and renewable energy is proposed, and the characteristics and scheduling arrangements of fuel cell vehicles are considered to further improve the local absorption …
Demand response (DR) for smart grids, which intends to balance the required power demand with the available supply resources, has been gaining widespread attention. The growing demand for electricity has presented new opportunities for residential load scheduling systems to improve energy consumption by shifting or curtailing the demand required with respect to price change or emergency cases. In this paper, a dynamic residential load scheduling system (DRLS) is proposed for optimal scheduling of household appliances on the basis of an adaptive consumption level (CL) pricing scheme (ACLPS). Th…
In the context of transition towards cleaner and sustainable energy production, microgrids have become an effective way for tackling environmental pollution and energy crisis issues. With the increasing penetration of renewables, how to coordinate demand response and renewable generations is a critical and challenging issue in the field of microgrid scheduling. To this end, a bi-level scheduling model is put forward for isolated microgrids with consideration of multi-stakeholders in this paper, where the lower- and upper-level models respectively aim to the minimization of user cost and microg…
In this work, a novel optimal scheduling approach is proposed for isolated microgrids (MGs) with renewable generations by incorporating demand response of electric vehicles (EVs). First, a bi-level programming-based MG scheduling model is proposed under real-time pricing environments, where the upper- and lower- levels seek to minimize the MG net operating cost and the EV charging cost. Second, a hybrid solution algorithm called JAYA-interior point method is put forward to solve the model. And finally, the simulation results demonstrate that incorporating demand response of electric vehicles i…
Distribution system integrated community microgrids (CMGs) can restore loads during extended outages. The CMG is challenged with limited resource availability, absence of a robust grid-support, and demand-supply uncertainty. To address these challenges, this paper proposes a three-stage hierarchical multi-timescale framework for scheduling and real-time (RT) dispatch of CMGs. The CMG's ability to dynamically expand its boundary to support the neighboring grid sections is also considered. The first stage solves a stochastic day-ahead (DA) scheduling problem to obtain referral plans for optimal …
Regarding the variability of renewable energy sources (RESs), especially in the operation time periods, their high penetration faces the net load pattern of the power system with a major ramp rate challenge. Although employing a market-based mechanism by the independent system operator (ISO) can address this challenge, it may not be possible to handle this challenge in all networks. In such networks, the required flexibility can be supplied by decreasing the ramp rate of the purchased power of distribution companies (Discos) from the market since their net load has an important impact on the s…
The control and managing of power demand and supply become very crucial because of penetration of renewables in the electricity networks and energy demand increase in residential and commercial sectors. In this paper, a new approach is presented to bridge the gap between Demand-Side Management (DSM) and microgrid portfolio, sizing and placement optimization. Although DSM helps energy consumers to take advantage of recent developments in utilization of Distributed Energy Resources (DERs) especially microgrids, a huge need of connecting DSM results to microgrid optimization is being felt. Conseq…
Northeast Electric Power University（东北电力大学）；State Grid Beijing Electric Power Company（国网北京市电力公司）
In order to reduce the negative impact of the uncertainty of load and renewable energies outputs on microgrid operation, an optimal scheduling model is proposed for isolated microgrids by using automated reinforcement learning-based multi-period forecasting of renewable power generations and loads. Firstly, a prioritized experience replay automated reinforcement learning (PER-AutoRL) is designed to simplify the deployment of deep reinforcement learning (DRL)-based forecasting model in a customized manner, the single-step multi-period forecasting method based on PER-AutoRL is proposed for the f…
In this paper, we investigate the management of energy storage control and load scheduling in scenarios considering a grid-connected photovoltaic (PV) system using packetized energy management. The aim is to reduce an average aggregated system cost through the proposed \textit{packetized energy management controller} considering household energy consumption, procurement price, load scheduling delays, PV self-sufficiency via generated renewable energy and battery degradation. The proposed approach solves the joint optimization problem using established heuristics, namely genetic algorithm (GA),…
Residential loads, especially heating, ventilation, and air conditioners (HVACs) and electric vehicles (EVs) have great potentials to provide demand flexibility which is an attribute of Grid-interactive Efficient Buildings (GEB). Under this new paradigm, first, EV and HVAC aggregator models are developed in this paper to represent the fleet of GEBs, in which the aggregated parameters are obtained based on a new approach of data generation and least-squares parameter estimation (DG-LSPE), which can deal with heterogenous HVACs. Then, a tri-level bidding and dispatching framework is established …
This paper proposes a reliable energy scheduling framework for distributed energy resources (DER) of a residential area to achieve an appropriate daily electricity consumption with the maximum affordable demand response. Renewable and non-renewable energy resources are available to respond to customers' demands using different classes of methods to manage energy during the time. The optimal operation problem is a mixed-integer-linear-programming (MILP) investigated using model-based predictive control (MPC) to determine which dispatchable unit should be operated at what time and at what power …
A bargaining game is investigated for cooperative energy management in microgrids. This game incorporates a fully distributed and realistic cooperative power scheduling algorithm (CoDES) as well as a distributed Nash Bargaining Solution (NBS)-based method of allocating the overall power bill resulting from CoDES. A novel weather-based stochastic renewable generation (RG) prediction method is incorporated in the power scheduling. We demonstrate the proposed game using a 4-user grid-connected microgrid model with diverse user demands, storage, and RG profiles and examine the effect of weather pr…
We consider the problem of energy management in microgrid networks. A microgrid is capable of generating a limited amount of energy from a renewable resource and is responsible for handling the demands of its dedicated customers. Owing to the variable nature of renewable generation and the demands of the customers, it becomes imperative that each microgrid optimally manages its energy. This involves intelligently scheduling the demands at the customer side, selling (when there is a surplus) and buying (when there is a deficit) the power from its neighboring microgrids depending on its current …
As multi-microgrids become readily available, some limited models have been proposed that study operational and power quality constraints with local energy markets independently. This paper proposes a convex optimization model of an energy management system with operational and power quality constraints and interactions in a Local Energy Market (LEM) for unbalanced microgrids (MGs). The LEM consists of a pre-dispatch step and an energy transactions step (ETS). The ETS combines the MGs' objectives while considering two strategies: minimize the cost of buyers or maximize the revenue of sellers. …
Argonne National Laboratory（阿贡国家实验室）；Oak Ridge National Laboratory（橡树岭国家实验室）
In this paper, we introduce a deep learning aided constraint encoding method to tackle the frequency-constraint microgrid scheduling problem. The nonlinear function between system operating condition and frequency nadir is approximated by using a neural network, which admits an exact mixed-integer formulation (MIP). This formulation is then integrated with the scheduling problem to encode the frequency constraint. With the stronger representation power of the neural network, the resulting commands can ensure adequate frequency response in a realistic setting in addition to islanding success. T…
Distribution system integrated community microgrids (CMGs) can partake in restoring loads during extended duration outages. At such times, the CMG is challenged with limited resource availability, absence of robust grid support, and heightened demand-supply uncertainty. This paper proposes a secure and adaptive three-stage hierarchical multi-timescale framework for scheduling and real-time (RT) dispatch of CMGs with hybrid PV systems to address these challenges. The framework enables the CMG to dynamically expand its boundary to support the neighboring grid sections and is adaptive to the chan…
In this report, several control strategies for microgrids with renewable energy resources and battery energy storage systems are proposed. Renewable energy has the potential to reduce global carbon emissions, while higher penetration of renewable energy is hindered by the inherent variability and intermittency of renewable resources. This motivates the development of microgrids supplied by renewable energy resources. With proper control of storage units and communications with the energy market, the non-dispatchable energy can be regulated to reduce the difficulty of power scheduling in the gr…
Department of Industrial Economics and Technology Management, The Norwegian University of Science and Technology（工业经济学与技术管理系，挪威科学与技术大学）
The electricity sector transition requires substantial increases in residential demand response capacity, yet Home Energy Management Systems (HEMS) adoption remains limited by user interaction barriers requiring translation of everyday preferences into technical parameters. While large language models have been applied to energy systems as code generators and parameter extractors, no existing implementation deploys LLMs as autonomous coordinators managing the complete workflow from natural language input to multi-appliance scheduling. This paper presents an agentic AI HEMS where LLMs autonomou…
This paper presents a novel grid-forming voltage control strategy for a battery energy storage system to maintain balanced three-phase output voltages when serving unbalanced loads. A stationary reference frame ($αβ$) based control scheme is proposed to regulate positive-sequence and negative-sequence voltages. Compared with the conventional rotating reference frame ($dq$) based control scheme, the proposed scheme shows better dynamic performance. Then, we analyze the system zero-sequence network and propose adding a grounding transformer to the Y/Yg connected output transformer to further red…
A microgrid is a new concept that has changed the power systems dramatically. It is a combination of Distributed Generation Resources (DGR) like Biomass, PV systems, Wind energy, Fuel cell, Diesel Generator, and so on with different types of loads (residential or commercial). Microgrids can work in two modes: autonomous and Interconnected. In the Islanding situation, the loads will be supported by DGR and without connecting to upstream utility grids. Controlling power electronic Interfaces between sources and loads has been an important task between the researchers. Several different strategie…
