The widespread adoption of photovoltaic (PV), electric vehicles (EVs), and stationary energy storage systems (ESS) in households increases system complexity while simultaneously offering new opportunities for energy regulation. However, effectively coordinating these resources under uncertainties remains challenging. This paper proposes a novel home energy management framework based on deep reinforcement learning (DRL) that can jointly minimise energy expenditure and battery degradation while guaranteeing occupant comfort and EV charging requirements. Distinct from existing studies, we explicitly account for the heterogeneous degradation characteristics of stationary and EV batteries in the optimisation, alongside stochastic user behaviour regarding arrival time, departure time, and driving distance. The energy scheduling problem is formulated as a constrained Markov decision process (CMDP) and solved using a Lagrangian soft actor-critic (SAC) algorithm. This approach enables the agent to learn optimal control policies that enforce physical constraints, including indoor temperature bounds and target EV state of charge upon departure, despite stochastic uncertainties. Numerical simulations over a one-year horizon demonstrate the effectiveness of the proposed framework in satisfying physical constraints while eliminating thermal oscillations and achieving significant economic benefits. Specifically, the method reduces the cumulative operating cost substantially compared to two standard rule-based baselines while simultaneously decreasing battery degradation costs by 8.44%.
Sapienza University of Rome（罗马第一大学）；IMDEA Energy Institute（IMDEA能源研究所）
Within electrical distribution networks, substation constraints management requires that aggregated power demand from residential users is kept within suitable bounds. Efficiency of substation constraints management can be measured as the reduction of constraints violations w.r.t. unmanaged demand. Home batteries hold the promise of enabling efficient and user-oblivious substation constraints management. Centralized control of home batteries would achieve optimal efficiency. However, it is hardly acceptable by users, since service providers (e.g., utilities or aggregators) would directly contr…
Dynamically reconfigurable batteries merge battery management with output formation in ac and dc batteries, increasing the available charge, power, and life time. However, the combined ripple generated by the load and the internal reconfiguration can degrade the battery. This paper introduces that the frequency range of the ripple matters for degradation and loss. It presents a novel control method that reduces the low-frequency ripple of dynamically reconfigurable battery technology to reduce cell ageing and loss. It furthermore shifts the residual ripple to higher frequencies where the lower…
Load flexibility management is a promising approach to face the problem of balancing generation and demand in electrical grids. This problem is becoming increasingly difficult due to the variability of renewable energies. Thermostatically controlled loads can be aggregated and managed by a virtual battery, and they provide a cost-effective and efficient alternative to physical storage systems to mitigate the inherent variability of renewable energy sources. But virtual batteries require of an accurate control system being capable of tracking frequency regulation signals with minimal error. A r…
Implementation of Linear Parameter Varying System to Investigate the Impact of Varying Flow Rate on the Lithium-ion Batteries Thermal Management System Performance
Electricity usage is a major portion of utility bills and the best place to start lowering them. An effective home energy management approach is introduced to decrease customers' electricity bills by determining the optimal appliance scheduling under hourly pricing based demand response (DR) strategies. The proposed approach specifically addresses consumer comfort through acceptable appliance deferral times, within which their consumption can be shifted from the normal schedules. Electric vehicles (EVs), with vehicle to grid (V2G) and vehicle-to-home (V2H) abilities, Energy storage systems (ES…
Whereas in typical standardized tests batteries are almost exclusively loaded with constant current or relatively slowly changing cycles, real applications involve rapid load ripple, which do not contribute to the net energy. The trend to reduced filter capacitors and even dynamically reconfigurable batteries further increases the ripple. The influence of rippled load on lithium batteries is therefore receiving increased attention. According to recent studies, accelerated ageing strongly depends on the frequency of the ripple. We use electrochemical models to derive a highly simplified regress…
In this paper, we investigate an energy cost minimization problem for a smart home in the absence of a building thermal dynamics model with the consideration of a comfortable temperature range. Due to the existence of model uncertainty, parameter uncertainty (e.g., renewable generation output, non-shiftable power demand, outdoor temperature, and electricity price) and temporally-coupled operational constraints, it is very challenging to determine the optimal energy management strategy for scheduling Heating, Ventilation, and Air Conditioning (HVAC) systems and energy storage systems in the sma…
Efficiency of Battery Energy Storage Systems (BESSs) is increasingly critical as renewable energy generation becomes more prevalent on the grid. Therefore, it is necessary to study the energy efficiency of lithium-ion batteries, which are typically used in BESSs. The purpose of this study is to propose the State of Efficiency (SOE) as a measure of how efficiently batteries transfer energy, and to analyze what factors affect the SOE of a battery throughout its lifetime. Using NASA's data set, we measure the SOE of Nickel-Cobalt-Aluminum (NCA) lithium-ion batteries by calculating the ratio of en…
A growing interest in the study of aging related phenomena in lithium-ion batteries is propelled by the increasing utilization of energy storage systems in electric vehicles and in buildings as stationery energy accumulators paired with renewable energy sources. This paper proposes a mixed-degradation model approach that combines the benefits of a semi-empirical approach with that of a physics-based model. This enables easy calibration for different battery chemistries, the ability to extrapolate when necessary, and is computationally efficient enough to be coupled with real-time running contr…
A neural network-based energy management system (NN-EMS) has been proposed in this paper for islanded ac microgrids fed by multiple PV-battery based distributed generators (DG). The stochastic and unequal irradiation results in unequal PV output, which causes an unequal state-of-charge (SoC) among the batteries of the DGs. This effect may cause the difference in the SoCs to increase considerably over time, leading to some batteries reaching their SoC limits. These batteries would no longer be able to control the dc-link of the hybrid grid forming DG. The proposed NN-EMS ensures SoC balancing b…
Internet of Things (IoT) and advanced communication technologies have demonstrated great potential to manage residential energy resources by enabling demand-side management (DSM). Home energy management systems (HEMSs) can automatically control electricity production and usage inside homes using DSM techniques. These HEMSs will wirelessly collect information from hardware installed in the power system and in homes with the objective to intelligently and efficiently optimize electricity usage and minimize costs. However, HEMSs can be vulnerable to cyberattacks that target the electricity pricin…
In this study, we investigate the operation of an optimal home energy management system (HEMS) with integrated renewable energy system (RES) and energy storage system (ESS) supporting electricity selling functions. A multi-objective mixed integer nonlinear programming model, including RES, ESS, home appliances and the main grid, is proposed to optimize different and conflicting objectives which are energy cost, user comfort and PAR. The effect of different selling prices on the objectives is also considered in detail. We further develop a formula for the lower bound of energy cost to help resi…
University of Central Florida（中佛罗里达大学）；University of Ottawa（渥太华大学）
A smart home energy management system (HEMS) can contribute towards reducing the energy costs of customers; however, HEMS suffers from uncertainty in both energy generation and consumption patterns. In this paper, we propose a sequence to sequence (Seq2Seq) learning-based supply and load prediction along with reinforcement learning-based HEMS control. We investigate how the prediction method affects the HEMS operation. First, we use Seq2Seq learning to predict photovoltaic (PV) power and home devices' load. We then apply Q-learning for offline optimization of HEMS based on the prediction resul…
Degradation-Aware Fast-Charging of Li-Ion Batteries Using Joint Electrical and Thermal Model Predictive Control
TNO Mobility and Built Environment（荷兰应用科学研究组织交通与环境部）；Eindhoven University of Technology, Department of Electrical Engineering（埃因霍温理工大学电气工程系）；National University of Science and Technology Politehnica Bucharest, Faculty of Electronics, Telecommunications and Information Technology（布加勒斯特国立理工大学电子、电信与信息技术学院）
Heating, ventilating, and air-conditioning (HVAC) systems consume a large amount of energy in residential houses and buildings. Effective energy management of HVAC is a cost-effective way to improve energy efficiency and reduce the energy cost of residential users. This work develops a novel distributed method for the residential transactive energy system that enables multiple users to interactively optimize their energy management of HVAC systems and behind-the-meter batteries. Specifically, this method effectively reduces the cost of smart homes by employing energy trading among users to lev…
With the booming of smart grid, The ubiquitously deployed smart meters constitutes an energy internet of things. This paper develops a novel blockchain-based transactive energy management system for IoT-aided smart homes. We consider a holistic set of options for smart homes to participate in transactive energy. Smart homes can interact with the grid to perform vertical transactions, e.g., feeding in extra solar energy to the grid and providing demand response service to alleviate the grid load. Smart homes can also interact with peer users to perform horizontal transactions, e.g., peer-to-pee…
Battery Second Life : A Review of Experimental Studies
Université Gustave Eiffel（巴黎-塞纳-圣但尼大学）；ENTPE（国立土木工程学院）；Universite Claude Bernard Lyon 1（里昂第一大学）；Ampère（安培研究所）；INSA Lyon（里昂国立应用科学学院）；Ecole Centrale de Lyon（里昂中央理工学院）；CNRS（法国国家科学研究中心）
With the increasing popularity of electric vehicles, distributed energy generation and storage facilities in smart grid systems, an efficient Demand-Side Management (DSM) is urgent for energy savings and peak loads reduction. Traditional DSM works focusing on optimizing the energy activities for a single household can not scale up to large-scale home energy management problems. Multi-agent Deep Reinforcement Learning (MA-DRL) shows a potential way to solve the problem of scalability, where modern homes interact together to reduce energy consumers consumption while striking a balance between en…
School of Electrical Engineering, Xi’an Jiaotong University（西安交通大学电气工程学院）
Accurate co-estimations of battery states, such as state-of-charge (SOC), state-of-health (SOH,) and remaining useful life (RUL), are crucial to the battery management systems to assure safe and reliable management. Although the external properties of the battery charge with the aging degree, batteries' degradation mechanism shares similar evolving patterns. Since batteries are complicated chemical systems, these states are highly coupled with intricate electrochemical processes. A state-coupled co-estimation method named Deep Inter and Intra-Cycle Attention Network (DIICAN) is proposed in thi…
Efficient residential sector coupling plays a key role in supporting the energy transition. In this study, we analyze the structural properties associated with the optimal control of a home energy management system and the effects of common technological configurations and objectives. We conduct this study by modeling a representative building with a modulating air-sourced heat pump, a photovoltaic (PV) system, a battery, and thermal storage systems for floor heating and hot-water supply. In addition, we allow grid feed-in by assuming fixed feed-in tariffs and consider user comfort. In our num…
State-of-Health Prediction for EV Lithium-Ion Batteries via DLinear and Robust Explainable Feature Selection
Department of Chemical Engineering（化学工程系）；Massachusetts Institute of Technology（麻省理工学院）；Department of Chemical and Biomolecular Engineering（化学与生物分子工程系）；Yonsei University（延世大学）；Department of Battery Engineering（电池工程系）
A novel centralized model predictive control (MPC) is proposed for comfort and energy management in a residential building. The residential setup used here is equipped with a photovoltaic (PV) solar system and a stationary home battery unit. An air-to-air multi-split heat pump (HP) is used as the primary heating system. The electric baseboard (BB) unit in each zone is used as a secondary system. The MPC is simultaneously responsible for controlling the heating inputs of the HP and BB units for comfort management, as well as for the control of energy flow between the PV, the home battery and th…
Optimal day-ahead scheduling for a system-centric community energy management system (CEMS) is proposed to provide economic benefits and user comfort of energy management at the community level. Our proposed community includes different types of homes and allows prosumers to trade energy locally using mid-market rate pricing. A mathematical model of the community is constructed and the optimization problem of this model is transformed into an MILP problem that can be solved in a short time. By solving this MILP problem, the optimization of the overall energy cost of the community and satisfact…
Parameter estimation is of foundational importance for various model-based battery management tasks, including charging control, state-of-charge estimation and aging assessment. However, it remains a challenging issue as the existing methods generally depend on cumbersome and time-consuming procedures to extract battery parameters from data. Departing from the literature, this paper sets the unique aim of identifying all the parameters offline in a one-shot procedure, including the resistance and capacitance parameters and the parameters in the parameterized function mapping from the state-of-…
Advanced battery management is to lithium-ion battery systems as the brain is to the human body. Its performance rests on the use of battery models that are both fast and accurate. However, mainstream equivalent circuit models and electrochemical models have yet to meet this need well, due to struggle with either predictive accuracy or computational complexity. This problem has acquired urgency as some emerging battery applications running across broad current ranges, e.g., electric vertical take-off and landing aircraft, can hardly find usable models from the literature. Motivated to address …
The need for resiliency of electricity supply is increasing due to increasing frequency of natural disasters---such as hurricanes---that disrupt supply from the power grid. Rooftop solar photovoltaic (PV) panels together with batteries can provide resiliency in many scenarios. Without intelligent and automated decision making that can trade off conflicting requirements, a large PV system and a large battery is needed to provide meaningful resiliency. By using forecast of solar generation and household demand, an intelligent decision maker can operate the equipment (battery and critical loads) …
