The development of renewable energy generation empowers microgrids to generate electricity to supply itself and to trade the surplus on energy markets. To minimize the overall cost, a microgrid must determine how to schedule its energy resources and electrical loads and how to trade with others. The control decisions are influenced by various factors, such as energy storage, renewable energy yield, electrical load, and competition from other microgrids. Making the optimal control decision is challenging, due to the complexity of the interconnected microgrids, the uncertainty of renewable energy generation and consumption, and the interplay among microgrids. The previous works mainly adopted the modeling-based approaches for deriving the control decision, yet they relied on the precise information of future system dynamics, which can be hard to obtain in a complex environment. This work provides a new perspective of obtaining the optimal control policy for distributed energy trading and scheduling by directly interacting with the environment, and proposes a multiagent deep reinforcement learning approach for learning the optimal control policy. Each microgrid is modeled as an agent, and different agents learn collaboratively for maximizing their rewards. The agent of each microgrid can make the local scheduling decision without knowing others' information, which can well maintain the autonomy of each microgrid. We evaluate the performances of our proposed method using real-world datasets. The experimental results show that our method can significantly reduce the cost of the microgrids compared with the baseline methods.
Demand Response (DR) has a widely recognized potential for improving grid stability and reliability while reducing customers energy bills. However, the conventional DR techniques come with several shortcomings, such as inability to handle operational uncertainties and incurring customer disutility, impeding their wide spread adoption in real-world applications. This paper proposes a new multiagent Reinforcement Learning (RL) based decision-making environment for implementing a Real-Time Pricing (RTP) DR technique in a prosumer dominated microgrid. The proposed technique addresses several short…
The emerging paradigm of interconnected microgrids advocates energy trading or sharing among multiple microgrids. It helps make full use of the temporal availability of energy and diversity in operational costs when meeting various energy loads. However, energy trading might not completely absorb excess renewable energy. A multi-energy management framework including fuel cell vehicles, energy storage, combined heat and power system, and renewable energy is proposed, and the characteristics and scheduling arrangements of fuel cell vehicles are considered to further improve the local absorption …
Integration of distributed energy resources (DERs) at a residential level has given way to the development of a self-sufficient energy-resilient system and peer-to-peer (P2P) energy trading at a community level. Several researchers have developed models to implement energy trading in the P2P network with DERs that enables prosumers to avail cost benefits. However, the impact of DERs and P2P energy network on the system's resilience is not evaluated with a quantifiable measure. In this work, we use the percolation threshold as a measure to quantify energy resilience, which can be computed using…
In recent years, there has been significant growth of distributed energy resources (DERs) penetration in the power grid. The stochastic and intermittent features of variable DERs such as roof top photovoltaic (PV) bring substantial uncertainties to the grid on the consumer end and weaken the grid reliability. In addition, the fact that numerous DERs are widespread in the grid makes it hard to monitor and manage DERs. To address this challenge, this paper proposes a novel real-time grid-supporting energy management (GSEM) strategy for grid-supporting microgrid (MG). This strategy can not only p…
Transactive microgrids have emerged as a transformative solution for the problems faced by distribution system operators due to an increase in the use of distributed energy resources and rapid growth in renewable energy generation. Transactive microgrids are tightly coupled cyber and physical systems, which require resilient and robust financial markets where transactions can be submitted and cleared, while ensuring that erroneous or malicious transactions cannot destabilize the grid. In this paper, we introduce TRANSAX, a novel decentralized platform for transactive microgrids. TRANSAX enable…
Keele University（基尔大学）；Luxembourg Institute of Science and Technology（卢森堡科学技术研究院）
In this paper, multi-agent reinforcement learning is used to control a hybrid energy storage system working collaboratively to reduce the energy costs of a microgrid through maximising the value of renewable energy and trading. The agents must learn to control three different types of energy storage system suited for short, medium, and long-term storage under fluctuating demand, dynamic wholesale energy prices, and unpredictable renewable energy generation. Two case studies are considered: the first looking at how the energy storage systems can better integrate renewable energy generation unde…
Energy Trading between microgrids Individual Cost Minimization and Social Welfare Maximization
Shanghai Jiao Tong University（上海交通大学）；Collaborative Innovation Center for Advanced Ship and Deep-Sea Exploration（先进船舶与深海探测协同创新中心）；Key Laboratory of System Control and Information Processing, Ministry of Education of China（教育部系统控制与信息处理重点实验室）；Shanghai Electric Group Co., Ltd Central Academe（上海电气集团中央研究院）
Efforts to utilize 100% renewable energy in community microgrids require new approaches to energy markets and transactions to efficiently address periods of scarce energy supply. In this paper we contribute to the promising approach of peer-to-peer (P2P) energy trading in two main ways: analysis of a centralized, welfare-maximizing economic dispatch that characterizes optimal price and allocations, and a novel P2P system for negotiating energy trades that yields physically feasible and at least weakly Pareto-optimal outcomes. Our main results are 1) that optimal pricing is insufficient to indu…
This paper develops an improved distributed finite-time control algorithm for multiagent-based ac microgrids with battery energy storage systems (BESSs) utilizing a low-width communication network. The proposed control algorithm can simultaneously coordinate BESSs to eliminate any deviation from the nominal frequency as well as solving the state of charge (SoC) balancing problem. The stability of the proposed control algorithm is established using the Lyapunov method and homogeneous approximation theory, which guarantees an accelerated convergence within a settling time that does not dependent…
With the increasing popularity of electric vehicles, distributed energy generation and storage facilities in smart grid systems, an efficient Demand-Side Management (DSM) is urgent for energy savings and peak loads reduction. Traditional DSM works focusing on optimizing the energy activities for a single household can not scale up to large-scale home energy management problems. Multi-agent Deep Reinforcement Learning (MA-DRL) shows a potential way to solve the problem of scalability, where modern homes interact together to reduce energy consumers consumption while striking a balance between en…
Peer-to-peer energy trading is emerging as a new paradigm that in the near future may disrupt conventional electricity markets and heavily affect energy exchanges in networks of microgrids. In this paper, a preference mechanism is considered to compute optimal energy exchanges in a network of microgrids with or without the supervision of the distribution system operator, and the alternating direction method of multipliers is adopted for its distributed solution. The effect of the preference mechanism on the resulting power flow in the network is further studied and discussed for realistic case…
Renewable energy-based microgrids play a critical role in future smart grids. Due to the uncertainties of renewable generations, the microgrids face potential risk of load shedding during operation. To address this problem, we propose a contract-based approach to enhance the resilience of microgrids. Specifically, in the framework, the microgrids who may not be self-efficient to meet their local demands can purchase the needed power from their connected microgrids by signing a contract that specifies the power price in advance. We leverage a principal-agent model to capture the energy trading …
In this paper we propose an original distributed control framework for DC mcirogrids. We first formulate the (optimal) control objectives as an aggregative game suitable for the energy trading market. Then, based on the dual theory, we analyze the equivalent distributed optimal condition for the proposed aggregative game and design a distributed control scheme to solve it. By interconnecting the DC mcirogrid and the designed distributed control system in a power preserving way, we steer the DC microgrid's state to the desired optimal equilibrium, satisfying a predefined set of local and coupli…
In this study, a cooperative game model is presented to schedule the day-ahead operation of multi-microgrid (MMG) systems. In the proposed model, microgrids are scheduled to achieve a global optimum for the cost of the multi-microgrid system. The minimum cost is achieved by transactions of microgrids with each other. Also, price-based demand response is implemented in the model to build a cost-reducing opportunity for consumers. Applying Shapley value, the optimum cost of the MMG system is fairly allocated between microgrids. To enhance the confidence level of results, data uncertainties are i…
We consider the problem of energy management in microgrid networks. A microgrid is capable of generating a limited amount of energy from a renewable resource and is responsible for handling the demands of its dedicated customers. Owing to the variable nature of renewable generation and the demands of the customers, it becomes imperative that each microgrid optimally manages its energy. This involves intelligently scheduling the demands at the customer side, selling (when there is a surplus) and buying (when there is a deficit) the power from its neighboring microgrids depending on its current …
To accommodate the advent of microgrids (MG) managing distributed energy resources (DER) in distribution systems, an interactive two-stage joint retail electricity market mechanism is proposed to provide an effective platform for these prosumers to proactively join in retail transactions. Day-ahead stochastic energy trading between the distribution system operator (DSO) and MGs is conducted in the first stage of a centralized retail market, where a chance-constrained uncertainty distribution locational marginal price (CC-UDLMP) containing the cost of uncertainty precautions is used to settle t…
Microgrids (MG) are anticipated to be important players in the future smart grid. For proper operation of MGs an Energy Management System (EMS) is essential. The EMS of an MG could be rather complicated when renewable energy resources (RER), energy storage system (ESS) and demand side management (DSM) need to be orchestrated. Furthermore, these systems may belong to different entities and competition may exist between them. Nash equilibrium is most commonly used for coordination of such entities however the convergence and existence of Nash equilibrium can not always be guaranteed. To this end…
This paper proposes a peer to peer (P2P), blockchain based energy trading market platform for residential communities with the objective of reducing overall community peak demand and household electricity bills. Smart homes within the community place energy bids for its available distributed energy resources (DERs) for each discrete trading period during a day, and a double auction mechanism is used to clear the market and compute the market clearing price (MCP). The marketplace is implemented on a permissioned blockchain infrastructure, where bids are stored to the immutable ledger and smart …
Most power systems are increasingly based on distributed energy resources, leading to a strong impact on the electrical grid management. In the case of Portugal, the contribution of renewable energy sources to the electricity generation portfolio is already high and the objective is to achieve 100% by 2050. Most of the new renewable generation capacity will be ensured by distributed photovoltaic generation installed in buildings and therefore the inherent intermittence of photovoltaic output combined with a mismatch with demand profile will challenge the operation and resiliency of the electri…
