基于强化学习的多个EV充电站需求响应协调的优化成本函数
Optimized cost function for demand response coordination of multiple EV charging stations using reinforcement learning
摘要:
电动汽车(EV)充电站代表了具有显著灵活性的大量负载。需求响应(DR)算法的灵活性的开发变得越来越重要的管理和平衡的需求和供应在电网。基于强化学习(RL)的无模型DR是平衡这种EV充电负载的有吸引力的方法。我们建立在以前的研究RL,基于马尔可夫决策过程(MDP),同时协调多个充电站。然而,我们注意到,在以前的研究中采用的计算昂贵的成本函数导致大量的训练时间,这限制了该方法的可行性和实用性。因此,我们提出了一种改进的成本函数,基本上迫使学习控制策略始终满足任何充电需求,不提供任何灵活性。我们严格比较新提出的批量RL拟合Q-迭代实现与原来的(昂贵的),使用真实世界的数据。具体而言,对于负载平坦化的情况,我们比较了两种方法的(i)处理时间来学习基于RL的计费策略,以及(ii)在满足目标负载方面的策略决策的整体性能看不见的测试数据。针对不同的训练周期和不同的训练样本大小分析性能。除了两个RL政策的性能结果,我们提供了性能界限方面(i)最佳的全知策略,和(ii)一个简单的启发式传播个人电动汽车充电均匀随着时间的推移
英文摘要:
Electric vehicle (EV) charging stations represent a substantial load with significant flexibility. The exploitation of that flexibility in demand response (DR) algorithms becomes increasingly important to manage and balance demand and supply in power grids. Model-free DR based on reinforcement learning (RL) is an attractive approach to balance such EV charging load. We build on previous research on RL, based on a Markov decision process (MDP) to simultaneously coordinate multiple charging stations. However, we note that the computationally expensive cost function adopted in the previous research leads to large training times, which limits the feasibility and practicality of the approach. We, therefore, propose an improved cost function that essentially forces the learned control policy to always fulfill any charging demand that does not offer any flexibility. We rigorously compare the newly proposed batch RL fitted Q-iteration implementation with the original (costly) one, using real-world data. Specifically, for the case of load flattening, we compare the two approaches in terms of (i) the processing time to learn the RL-based charging policy, as well as (ii) the overall performance of the policy decisions in terms of meeting the target load for unseen test data. The performance is analyzed for different training periods and varying training sample sizes. In addition to both RL policies performance results, we provide performance bounds in terms of both (i) an optimal all-knowing strategy, and (ii) a simple heuristic spreading individual EV charging uniformly over time
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。