arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.07500eess.SYcs.SY
收藏

强化学习在区域供冷系统优化控制中的应用

Reinforcement Learning for Optimal Control of a District Cooling Energy Plant

Zhong Guo, Austin R. Coffman, Prabir Barooah

备注：
10 pages, extended ACC2022 version
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

摘要：

区域冷却能源工厂（DCEP）由冷水机组，冷却塔和热能储存（TES）系统组成，消耗大量的电力。利用时变电价优化TES和冷水机组的调度是一个具有挑战性的最优控制问题。传统的模型预测控制（MPC）方法需要求解高维混合整数非线性规划（MINLP），因为制冷机的开/关驱动和TES的充/放电在计算上具有挑战性。RL是一个有吸引力的替代MPC：实时控制计算是一个低维的优化问题，可以很容易地解决。然而，RL控制器的性能取决于许多设计选择。在本文中，我们提出了一个基于Q学习的强化学习（RL）控制器的这个问题。数值模拟结果表明，建议的RL控制器是能够减少能源成本超过基于规则的基线控制器约8%，与文献中报道的节省与MPC类似的DCEP。我们描述了RL控制器的设计选择，包括基函数，奖励函数成形和学习算法参数。与现有的DCEP强化学习的工作相比，所提出的控制器是针对连续的状态和动作空间设计的。

英文摘要：

District cooling energy plants (DCEPs) consisting of chillers, cooling towers, and thermal energy storage (TES) systems consume a considerable amount of electricity. Optimizing the scheduling of the TES and chillers to take advantage of time-varying electricity price is a challenging optimal control problem. The classical method, model predictive control (MPC), requires solving a high dimensional mixed-integer nonlinear program (MINLP) because of the on/off actuation of the chillers and charging/discharging of TES, which are computationally challenging. RL is an attractive alternative to MPC: the real time control computation is a low-dimensional optimization problem that can be easily solved. However, the performance of an RL controller depends on many design choices. In this paper, we propose a Q-learning based reinforcement learning (RL) controller for this problem. Numerical simulation results show that the proposed RL controller is able to reduce energy cost over a rule-based baseline controller by approximately 8%, comparable to savings reported in the literature with MPC for similar DCEPs. We describe the design choices in the RL controller, including basis functions, reward function shaping, and learning algorithm parameters. Compared to existing work on RL for DCEPs, the proposed controller is designed for continuous state and actions spaces.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑