arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2212.06001eess.SYcs.LGcs.SY
收藏

机组组合问题的强化学习和树搜索方法

Reinforcement Learning and Tree Search Methods for the Unit Commitment Problem

Bartlett School of Environment, Energy and Resources(巴特利特环境、能源与资源学院)

Patrick de Mars

摘要:

机组组合问题是电力系统运行中的一项基本任务,它决定着满足电力需求的机组运行计划。现有的UC方法使用混合整数规划不太适合高度随机系统。更严格地考虑不确定性的方法可以通过降低旋转储备要求,以更高的效率运营发电站,以及整合更大数量的可变可再生能源来大幅降低运营成本。解决UC问题的一种有前途的方法是强化学习(RL),这是一种用于最优决策的方法,已被用于克服人工智能中长期存在的重大挑战。本论文探讨强化学习在UC问题中的应用,并解决了包括不确定性下的鲁棒性、多个问题实例的可推广性以及扩展到比以前研究的更大的电力系统的挑战。为了解决这些问题,我们开发了引导树搜索,这是一种结合无模型RL和基于模型的规划的新方法。UC问题被正式化为马尔可夫决策过程,我们开发了一个开源环境的基础上,从英国的电力系统的真实数据训练RL代理。在多达100个发电机的问题中,引导树搜索被证明是具有竞争力的确定性UC方法,降低运营成本高达1.4%。RL的一个优点是,该框架可以很容易地扩展到纳入考虑重要的电力系统运营商,如鲁棒性发电机故障,风力削减或碳价格。当考虑发电机停运时,与使用传统的$N-x$备用标准的方法相比,引导树搜索节省了2%以上的运行成本。

英文摘要:

The unit commitment (UC) problem, which determines operating schedules of generation units to meet demand, is a fundamental task in power systems operation. Existing UC methods using mixed-integer programming are not well-suited to highly stochastic systems. Approaches which more rigorously account for uncertainty could yield large reductions in operating costs by reducing spinning reserve requirements; operating power stations at higher efficiencies; and integrating greater volumes of variable renewables. A promising approach to solving the UC problem is reinforcement learning (RL), a methodology for optimal decision-making which has been used to conquer long-standing grand challenges in artificial intelligence. This thesis explores the application of RL to the UC problem and addresses challenges including robustness under uncertainty; generalisability across multiple problem instances; and scaling to larger power systems than previously studied. To tackle these issues, we develop guided tree search, a novel methodology combining model-free RL and model-based planning. The UC problem is formalised as a Markov decision process and we develop an open-source environment based on real data from Great Britain's power system to train RL agents. In problems of up to 100 generators, guided tree search is shown to be competitive with deterministic UC methods, reducing operating costs by up to 1.4\%. An advantage of RL is that the framework can be easily extended to incorporate considerations important to power systems operators such as robustness to generator failure, wind curtailment or carbon prices. When generator outages are considered, guided tree search saves over 2\% in operating costs as compared with methods using conventional $N-x$ reserve criteria.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑