arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2306.08147cs.LG
收藏

基于强化学习的含可再生能源多市场能源优化

Multi-market Energy Optimization with Renewables via Reinforcement Learning

Lucien Werner, Peeyush Kumar

AI总结:

本文提出深度强化学习框架,将含储能的可再生能源电厂多市场运营优化建模为分层MDP,通过动作投影满足约束,适配复杂储能模型,经美印市场数据验证效果优于基线策略。

摘要:

本文提出一种深度强化学习(RL)框架,用于优化配备储能的可再生能源发电厂的运营。其目标是最大化能源市场收益,同时最小化储能损耗成本与可再生能源弃电量。该框架可处理储能设备带来的时间耦合、可再生能源发电量与能源价格的不确定性,以及非线性储能模型等复杂问题。本研究将该问题建模为分层马尔可夫决策过程(MDP),并采用组件级储能模拟器。框架利用RL整合复杂储能模型,突破了基于优化的方法需依赖凸性、可微组件模型的限制。该方法的核心亮点是通过一种将潜在不可行动作投影到安全状态-动作集合的新方法,确保策略动作满足系统约束。本文利用美国和印度电力市场数据开展大量实验,将习得的RL策略与基线控制策略、回溯最优控制策略进行对比,验证了该方法的有效性。研究还验证了该学习框架适配多种储能模型,证明了RL在多市场竞价、概率预测与精准储能组件模型场景下,复杂能源优化问题中的应用价值。

英文摘要:

This paper introduces a deep reinforcement learning (RL) framework for optimizing the operations of power plants pairing renewable energy with storage. The objective is to maximize revenue from energy markets while minimizing storage degradation costs and renewable curtailment. The framework handles complexities such as time coupling by storage devices, uncertainty in renewable generation and energy prices, and non-linear storage models. The study treats the problem as a hierarchical Markov Decision Process (MDP) and uses component-level simulators for storage. It utilizes RL to incorporate complex storage models, overcoming restrictions of optimization-based methods that require convex and differentiable component models. A significant aspect of this approach is ensuring policy actions respect system constraints, achieved via a novel method of projecting potentially infeasible actions onto a safe state-action set. The paper demonstrates the efficacy of this approach through extensive experiments using data from US and Indian electricity markets, comparing the learned RL policies with a baseline control policy and a retrospective optimal control policy. It validates the adaptability of the learning framework with various storage models and shows the effectiveness of RL in a complex energy optimization setting, in the context of multi-market bidding, probabilistic forecasts, and accurate storage component models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑