Reinforcement learning (RL) and model predictive control (MPC) each offer distinct advantages and limitations when applied to control problems in power and energy systems. Despite various studies on these methods, benchmarks remain lacking and the preference for RL over traditional controls is not well understood. In this work, we put forth a comparative analysis using RL- and MPC-based controllers for optimizing a battery management system (BMS). The BMS problem aims to minimize costs while adhering to operational limits. by adjusting the battery (dis)charging in response to fluctuating electricity prices over a time horizon. The MPC controller uses a learningbased forecast of future demand and price changes to formulate a multi-period linear program, that can be solved using off-the-shelf solvers. Meanwhile, the RL controller requires no timeseries modeling but instead is trained from the sample trajectories using the proximal policy optimization (PPO) algorithm. Numerical tests compare these controllers across optimality, training time, testing time, and robustness, providing a comprehensive evaluation of their efficacy. RL not only yields optimal solutions quickly but also ensures robustness to shifts in customer behavior, such as changes in demand distribution. However, as expected, training the RL agent is more time-consuming than MPC.
Battery Asset Management problem determines the minimum cost replacement schedules for each individual asset in a group of battery assets that operate in parallel. Battery cycle life varies under different operating conditions including temperature, depth of discharge, charge rate, etc., and a battery deteriorates due to usage, which cannot be handled by current asset management models. This paper presents battery cycle life prognosis and its integration with parallel asset management to reduce lifecycle cost of the Battery Energy Storage System (BESS). A nonlinear capacity fade model is incor…
Battery management system plays an important role for modern battery-powered application such as Electric vehicles, portable electronic equipment and storage for renewable energy sources. It also increases the life-cycle of the battery, battery state and efficiency. Monitoring the state of charge of the battery is a crucial factor for battery management system. This paper deals with monitoring the state of charge of the battery along with temperature, current for Solar panel fitted with battery for residential application. Microcontroller is used for controlling purpose, analog sensors are use…
This article proposes an offline Energy Management System (EMS) for Parallel Hybrid Electric Vehicles (PHEVs). Dividing the torque between the Electric Motor (EM) and the Internal Combustion Engine (ICE) requires a suitable EMS. Batteries are vital to HEVs and significantly impact overall vehicle cost and performance. High temperature and high battery State of Charge (SOC) are the main factors that accelerate battery aging. SOC is the most critical state variable in EMS and was usually considered the only dynamic variable in previous studies. For simplicity, the battery temperature was often a…
Battery swapping as a business model for battery energy storage (BES) has great potential in future integrated low-carbon energy and transportation systems. However, frequent battery swapping will inevitably accelerate battery degradation and shorten the battery life accordingly. To model the tradeoff of BES use between energy and transportation applications coupled by battery swapping, we develop a life-cycle decision model that coordinates battery charging and swapping. This model is derived based on an improved intertemporal decision framework, in which the optimal marginal degradation cost…
A novel smart metering technique capable of anomaly detection was proposed for real-time home power management system. Smart meter data generated in real-time was obtained from 900 households of single apartments. To detect outliers and missing values in smart meter data, a deep learning model, the autoencoder, consisting of a graph convolutional network and bidirectional long short-term memory network, was applied to the smart metering technique. Power management based on the smart metering technique was performed by multi-objective optimization in the presence of a battery storage system and…
This paper addresses optimal battery thermal management (BTM), charging, and eco-driving of a battery electric vehicle (BEV) with the goal of improving its grid-to-meter energy efficiency. Thus, an optimisation problem is formulated, aiming at finding the optimal trade-off between trip time and charging cost. The formulated problem is then transformed into a hybrid dynamical system, where the dynamics in driving and charging modes are modeled with different functions and with different state and control vectors. Moreover, to improve computational efficiency, we propose modelling the driving dy…
This paper proposes a deep learning-based optimal battery management scheme for frequency regulation (FR) by integrating model predictive control (MPC), supervised learning (SL), reinforcement learning (RL), and high-fidelity battery models. By taking advantage of deep neural networks (DNNs), the derived DNN-approximated policy is computationally efficient in online implementation. The design procedure of the proposed scheme consists of two sequential processes: (1) the SL process, in which we first run a simulation with an MPC embedding a low-fidelity battery model to generate a training data…
In this paper, we present the use of Model Predictive Control (MPC) based on Reinforcement Learning (RL) to find the optimal policy for a multi-agent battery storage system. A time-varying prediction of the power price and production-demand uncertainty are considered. We focus on optimizing an economic objective cost while avoiding very low or very high state of charge, which can damage the battery. We consider the bounded power provided by the main grid and the constraints on the power input and state of each agent. A parametrized MPC-scheme is used as a function approximator for the determin…
Hybrid fast-charging stations with battery storage and local renewable generation can facilitate low-carbon electric vehicle (EV) charging, while reducing the stress on the distribution grid. This paper proposes energy management strategies for a novel multi-battery design that directly connects its strings to other DC components through a busbar matrix without the need for interfacing power converters. Hence, the energy management system has two degrees of control: (i) allocating strings to other DC microgrid components, in this case a photovoltaic system, two EV fast chargers, and a grid-tie…
We optimize the operation of a fuel cell hybrid train using convex optimization. The main objective is to minimize hydrogen fuel consumption for a target journey time while considering battery thermal constraints. The state trajectories: train speed, energy management system, and battery temperature, are all optimized concurrently within a single optimization problem. A novel thermal model is proposed in order to include battery temperature yet maintain formulation convexity. Simulations show fuel savings and better thermal management when temperature is optimized concurrently with the other s…
A full-time scale energy management and battery size optimization for off-grid renewable power to hydrogen systems: A battery energy storage-based grid-forming case in Inner Mongolian
Monitoring the state of health for Li-ion batteries is crucial in the battery management system (BMS), which helps end-users use batteries efficiently and safely. Battery state of health can be monitored by identifying parameters of battery models using various algorithms. Due to the low computation power of BMS and time-varying parameters, it is very important to develop an online algorithm with low computational cost. Among various methods, Equivalent circuit model (ECM) -based recursive least squares (RLS) parameter identification is well suited for such difficult BMS environments. However,…
Fuel cell electric vehicles have earned substantial attentions in recent decades due to their high-efficiency and zero-emission features, while the high operating costs remain the major barrier towards their large-scale commercialization. In such context, this paper aims to devise an energy management strategy for an urban postal-delivery fuel cell electric vehicle for operating cost mitigation. First, a data-driven dual-loop spatial-domain battery state-of-charge reference estimator is designed to guide battery energy depletion, which is trained by real-world driving data collected in postal …
Whereas in typical standardized tests batteries are almost exclusively loaded with constant current or relatively slowly changing cycles, real applications involve rapid load ripple, which do not contribute to the net energy. The trend to reduced filter capacitors and even dynamically reconfigurable batteries further increases the ripple. The influence of rippled load on lithium batteries is therefore receiving increased attention. According to recent studies, accelerated ageing strongly depends on the frequency of the ripple. We use electrochemical models to derive a highly simplified regress…
Dynamically reconfigurable batteries merge battery management with output formation in ac and dc batteries, increasing the available charge, power, and life time. However, the combined ripple generated by the load and the internal reconfiguration can degrade the battery. This paper introduces that the frequency range of the ripple matters for degradation and loss. It presents a novel control method that reduces the low-frequency ripple of dynamically reconfigurable battery technology to reduce cell ageing and loss. It furthermore shifts the residual ripple to higher frequencies where the lower…
A neural network-based energy management system (NN-EMS) has been proposed in this paper for islanded ac microgrids fed by multiple PV-battery based distributed generators (DG). The stochastic and unequal irradiation results in unequal PV output, which causes an unequal state-of-charge (SoC) among the batteries of the DGs. This effect may cause the difference in the SoCs to increase considerably over time, leading to some batteries reaching their SoC limits. These batteries would no longer be able to control the dc-link of the hybrid grid forming DG. The proposed NN-EMS ensures SoC balancing b…
Lithium-ion battery packs are usually composed of hundreds of cells arranged in series and parallel connections. The proper functioning of these complex devices requires suitable Battery Management Systems (BMSs). Advanced BMSs rely on mathematical models to assure safety and high performance. While many approaches have been proposed for the management of single cells, the control of multiple cells has been less investigated and usually relies on simplified models such as equivalent circuit models. This paper addresses the management of a battery pack in which each cell is explicitly modelled …
This paper studies optimal thermal management and charging of a battery electric vehicle driving over long distance trips. The focus is on the potential benefits of including a heat pump in the thermal management system for waste heat recovery, and charging point planning, in a way to achieve optimality in time, energy, or their trade-off. An optimal control problem is formulated, in which the objective function includes the energy delivered by the charger(s), and the total charging time including the actual charging time and the detour time to and from the charging stop. To reduce the computa…
A novel centralized model predictive control (MPC) is proposed for comfort and energy management in a residential building. The residential setup used here is equipped with a photovoltaic (PV) solar system and a stationary home battery unit. An air-to-air multi-split heat pump (HP) is used as the primary heating system. The electric baseboard (BB) unit in each zone is used as a secondary system. The MPC is simultaneously responsible for controlling the heating inputs of the HP and BB units for comfort management, as well as for the control of energy flow between the PV, the home battery and th…
Heriot-Watt University（赫瑞-瓦特大学）；University of Liverpool（利物浦大学）
The battery is a key component of autonomous robots. Its performance limits the robot's safety and reliability. Unlike liquid-fuel, a battery, as a chemical device, exhibits complicated features, including (i) capacity fade over successive recharges and (ii) increasing discharge rate as the state of charge (SOC) goes down for a given power demand. Existing formal verification studies of autonomous robots, when considering energy constraints, formalise the energy component in a generic manner such that the battery features are overlooked. In this paper, we model an unmanned aerial vehicle (UAV)…
Battery energy storage systems (ESS) are widely used in microgrids to complement high renewables. However, the real-time energy management of microgrids with battery ESS is challenging in two aspects: 1) the evolution process of battery energy level is across-time coupled; 2) uncertainties unavoidably arise in the forecasting process for renewable generation. In this paper, a tube-based model predictive control (MPC) approach is innovatively proposed in accommodating the real-time energy management of microgrids with battery ESS. Firstly, a real-time operation model of battery, including the d…
This paper presents a novel modular, reconfigurable battery energy storage system. The proposed design is characterized by a tight integration of reconfigurable power switches and DC/DC converters. This characteristic enables isolation of faulty cells from the system and allows fine power control for individual cells toward optimal system-level performance. An optimal power management approach is developed to extensively exploit the merits of the proposed design. Based on receding-horizon convex optimization, this approach aims to minimize the total power losses in charging/discharging while a…
A growing interest in the study of aging related phenomena in lithium-ion batteries is propelled by the increasing utilization of energy storage systems in electric vehicles and in buildings as stationery energy accumulators paired with renewable energy sources. This paper proposes a mixed-degradation model approach that combines the benefits of a semi-empirical approach with that of a physics-based model. This enables easy calibration for different battery chemistries, the ability to extrapolate when necessary, and is computationally efficient enough to be coupled with real-time running contr…
