arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1910.04541eess.SYcs.SY
收藏

微电网储能随机调度的增强强化学习方法

Stochastic Dispatch of Energy Storage in Microgrids: An Augmented Reinforcement Learning Approach

Yuwei Shang, Wenchuan Wu, Jianbo Guo, Zhe Lv, Zhao Ma, Wanxing Sheng, Ran Chen

摘要:

微电网中蓄电池储能系统的动态调度本质上是一个多周期随机优化问题。由于BESS的寿命受到其充电和放电行为的显著影响,因此其生命周期退化成本应纳入BESS的DD模型中,这使得它是非凸的。一般来说,这种MSOP是棘手的。为了解决这个问题,我们提出了一个强化学习(RL)的解决方案,增加了蒙特卡洛树搜索(MCTS)和领域知识表示为调度规则。在这个解决方案中,Q学习函数逼近作为基本的学习架构,允许多步自举和连续的策略学习。为了提高随机多步模拟的计算效率,我们采用MCTS来估计期望的最大动作值。此外,我们在强化学习中嵌入了一些调度规则作为概率逻辑,以减少不可行的动作探索,这可以提高数据驱动解决方案的质量。数值试验结果表明,该算法优于其他基线RL算法在所有情况下测试。

英文摘要:

The dynamic dispatch (DD) of battery energy storage systems (BESSs) in microgrids integrated with volatile energy resources is essentially a multiperiod stochastic optimization problem (MSOP). Because the life span of a BESS is significantly affected by its charging and discharging behaviors, its lifecycle degradation costs should be incorporated into the DD model of BESSs, which makes it non-convex. In general, this MSOP is intractable. To solve this problem, we propose a reinforcement learning (RL) solution augmented with Monte-Carlo tree search (MCTS) and domain knowledge expressed as dispatching rules. In this solution, the Q-learning with function approximation is employed as the basic learning architecture that allows multistep bootstrapping and continuous policy learning. To improve the computation efficiency of randomized multistep simulations, we employed the MCTS to estimate the expected maximum action values. Moreover, we embedded a few dispatching rules in RL as probabilistic logics to reduce infeasible action explorations, which can improve the quality of the data-driven solution. Numerical test results show the proposed algorithm outperforms other baseline RL algorithms in all cases tested.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑