This paper introduces a deep reinforcement learning (RL) framework for optimizing the operations of power plants pairing renewable energy with storage. The objective is to maximize revenue from energy markets while minimizing storage degradation costs and renewable curtailment. The framework handles complexities such as time coupling by storage devices, uncertainty in renewable generation and energy prices, and non-linear storage models. The study treats the problem as a hierarchical Markov Decision Process (MDP) and uses component-level simulators for storage. It utilizes RL to incorporate complex storage models, overcoming restrictions of optimization-based methods that require convex and differentiable component models. A significant aspect of this approach is ensuring policy actions respect system constraints, achieved via a novel method of projecting potentially infeasible actions onto a safe state-action set. The paper demonstrates the efficacy of this approach through extensive experiments using data from US and Indian electricity markets, comparing the learned RL policies with a baseline control policy and a retrospective optimal control policy. It validates the adaptability of the learning framework with various storage models and shows the effectiveness of RL in a complex energy optimization setting, in the context of multi-market bidding, probabilistic forecasts, and accurate storage component models.
Real-time scheduling of renewable power systems through planning-based reinforcement learning
Tsinghua University(清华大学);State Grid Corporation of China(国家电网有限公司);National Power Dispatching and Control Center(国家电力调度控制中心);China Electric Power Research Institute(中国电力科学研究院)
Improving Sequential Market Coordination via Value-oriented Renewable Energy Forecasting
College of Engineering, Cornell University(康奈尔大学工程学院);Department of Electrical and Computer Engineering, University of California, San Diego(加州大学圣地亚哥分校电气与计算机工程系);Department of Electrical Engineering, Shanghai Jiao Tong University(上海交通大学电气工程系);Dyson School of Design Engineering, Imperial College London(伦敦帝国理工学院设计工程学院)
Value-oriented forecast reconciliation for renewables in electricity markets
Department of Electrical Engineering, Shanghai Jiao Tong University, China(上海交通大学电气工程系);Dyson School of Design Engineering, Imperial College London, United Kingdom(帝国理工学院伦敦戴森设计工程学院);Department of Technology, Management and Economics, Technical University of Denmark, Denmark(丹麦技术大学技术、管理与经济系);CoRE, Aarhus University, Denmark(奥胡斯大学CoRE)