利用深度强化学习和时间序列预测增强电池储能能源套利
Enhancing Battery Storage Energy Arbitrage with Deep Reinforcement Learning and Time-Series Forecasting
AI总结：
针对电池储能能源套利中的电价不确定性，本文将 DRL 与多时间范围时间序列预测结合，并在阿尔伯塔电价数据上验证 DQN 累积奖励提升 60%。
摘要：
能源套利是电池运营商最有利可图的收入来源之一，通过在不同电价下买卖电力来产生收益。由于电价固有的不确定性，预测这些收益颇具挑战。近年来，深度强化学习（DRL）作为一种很有前景的工具出现，能够通过在大量历史数据上训练来应对不确定性。然而，在无法获取未来电价的情况下，DRL 智能体只能对当前观测到的价格作出反应，而无法学习规划电池调度。因此，在本研究中，我们将 DRL 与来自深度学习的时间序列预测方法相结合，以提升能源套利方面的表现。我们使用加拿大阿尔伯塔省的电价数据开展案例研究，该数据的特点是具有不规则的价格尖峰且高度非平稳。即使部署由卷积层、循环层和注意力模块组成的最先进深度学习模型，这类数据也很难预测。我们的结果表明，具备 DRL 能力的电池控制所实现的能源套利仍会显著受益于这些并不完美的预测，但前提是必须组合多个时间范围的预测器。将未来 24 小时窗口的多项预测分组后，与没有预测的实验相比，深度 Q 网络（DQN）的累积奖励提高了 60%。我们假设，多个预测器尽管存在不完美之处，却能通过“多数投票”原则传达有关电价未来走势的有用信息，从而使 DRL 智能体能够学习到更有利可图的控制策略。
英文摘要：
Energy arbitrage is one of the most profitable sources of income for battery operators, generating revenues by buying and selling electricity at different prices. Forecasting these revenues is challenging due to the inherent uncertainty of electricity prices. Deep reinforcement learning (DRL) emerged in recent years as a promising tool, able to cope with uncertainty by training on large quantities of historical data. However, without access to future electricity prices, DRL agents can only react to the currently observed price and not learn to plan battery dispatch. Therefore, in this study, we combine DRL with time-series forecasting methods from deep learning to enhance the performance on energy arbitrage. We conduct a case study using price data from Alberta, Canada that is characterized by irregular price spikes and highly non-stationary. This data is challenging to forecast even when state-of-the-art deep learning models consisting of convolutional layers, recurrent layers, and attention modules are deployed. Our results show that energy arbitrage with DRL-enabled battery control still significantly benefits from these imperfect predictions, but only if predictors for several horizons are combined. Grouping multiple predictions for the next 24-hour window, accumulated rewards increased by 60% for deep Q-networks (DQN) compared to the experiments without forecasts. We hypothesize that multiple predictors, despite their imperfections, convey useful information regarding the future development of electricity prices through a "majority vote" principle, enabling the DRL agent to learn more profitable control policies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
