Optimal control of thermostatically controlled loads connected to a district heating network is considered a sequential decision- making problem under uncertainty. The practicality of a direct model-based approach is compromised by two challenges, namely scalability due to the large dimensionality of the problem and the system identification required to identify an accurate model. To help in mitigating these problems, this paper leverages on recent developments in reinforcement learning in combination with a market-based multi-agent system to obtain a scalable solution that obtains a significant performance improvement in a practical learning time. The control approach is applied on a scenario comprising 100 thermostatically controlled loads connected to a radial district heating network supplied by a central combined heat and power plant. Both for an energy arbitrage and a peak shaving objective, the control approach requires 60 days to obtain a performance within 65% of a theoretical lower bound on the cost.
At high latitudes, many cities adopt a centralized heating system to improve the energy generation efficiency and to reduce pollution. In multi-tier systems, so-called district heating, there are a few efficient approaches for the flow rate control during the heating process. In this paper, we describe the theoretical methods to solve this problem by deep reinforcement learning and propose a cloud-based heating control system for implementation. A real-world case study shows the effectiveness and practicability of the proposed system controlled by humans, and the simulated experiments for deep…
Achieving efficiency gains in Chinese district heating networks, thereby reducing their carbon footprint, requires new optimal control methods going beyond current industry tools. Focusing on the secondary network, we propose a data-driven deep reinforcement learning (DRL) approach to address this task. We build a recurrent neural network, trained on simulated data, to predict the indoor temperatures. This model is then used to train two DRL agents, with or without expert guidance, for the optimal control of the supply water temperature. Our tests in a multi-apartment setting show that both ag…
KU Leuven/EnergyVille（鲁汶大学/能源城）；REstore（REstore公司）；Vrije Universiteit Brussel（布鲁塞尔自由大学）；Vito/EnergyVille（弗拉芒技术研究院/能源城）
This paper considers a demand response agent that must find a near-optimal sequence of decisions based on sparse observations of its environment. Extracting a relevant set of features from these observations is a challenging task and may require substantial domain knowledge. One way to tackle this problem is to store sequences of past observations and actions in the state vector, making it high dimensional, and apply techniques from deep learning. This paper investigates the capabilities of different deep learning techniques, such as convolutional neural networks and recurrent neural networks,…
Demand response (DR) programs aim to engage distributed small-scale flexible loads, such as thermostatically controllable loads (TCLs), to provide various grid support services. Linearly Solvable Markov Decision Process (LS-MDP), a variant of the traditional MDP, is used to model aggregated TCLs. Then, a model-free reinforcement learning technique called Z-learning is applied to learn the value function and derive the optimal policy for the DR aggregator to control TCLs. The learning process is robust against uncertainty that arises from estimating the passive dynamics of the aggregated TCLs. …
Modern district heating technologies have a great potential to make the energy sector more flexible and sustainable due to their capabilities to use energy sources of varied nature and to efficiently store energy for subsequent use. Central control tasks within these systems for the efficient and safe distribution of heat refer to the stabilization of overall system temperatures and the regulation of storage units state of charge. These are challenging goals when the networked and nonlinear nature of district heating system models is taken into consideration. In this letter, for district heati…
Flow and storage volume regulation is essential for the adequate transport and management of energy resources in district heating systems. In this letter, we propose a novel and suitably tailored -- decentralized -- adaptive control scheme addressing this problem whilst offering closed-loop stability guarantees. We focus on a system configuration comprising multiple heat producers, consumers and storage tanks exchanging energy through a common distribution network, which are features of modern and prospective district heating installations. {\color{black}The proposed controller is based on pas…
The flexible power consumption feature of thermostatically controlled loads (TCLs) such as heating, ventilation, and air-conditioning (HVAC) systems makes them attractive targets for demand response (DR). TCLs possess a brief period where their power utilization can be altered without any significant impact on customer comfort level. This indicates TCLs are hidden potentials for providing ancillary services. This paper proposes a novel metric of demand response support time (DRST) for HVAC enabled demand response and a novel algorithm for the quantification of such HVAC-DR. The consumers' comf…
Massive adoptions of combined heat and power (CHP) units necessitate the coordinated operation of power system and district heating system (DHS). Exploiting the reconfigurable property of district heating networks (DHNs) provides a cost-effective solution to enhance the flexibility of the power system by redistributing heat loads in DHS. In this paper, a unit commitment considering combined electricity and reconfigurable heating network (UC-CERHN) is proposed to coordinate the day-ahead scheduling of power system and DHS. The DHS is formulated as a nonlinear and mixed-integer model with consid…
We propose a comprehensive nonlinear ODE-based thermo-hydraulic model of a district heating system featuring several heat producers, consumers and storage devices which are interconnected through a distribution network of meshed topology whose temperature dynamics are explicitly considered. Moreover, we analyze the conditions under which the hydraulic and thermal subsystems of the model exhibit shifted passivity properties. For the hydraulic subsystem, our claims on passivity draw on the monotonicity of the vector field associated to the DH system's flow dynamics, which mainly codifies viscous…
Poznan University of Technology（波兹南理工大学）；Kogeneracja Zachód S.A.（西部热电股份公司）
In this paper, we consider energy demand prediction in district heating systems. Effective energy demand prediction is essential in combined heat power systems when offering electrical energy in competitive electricity markets. To address this problem, we propose two sets of algorithms: (1) a novel extension to the algorithm proposed by E. Dotzauer and (2) an autoregressive predictor based on hour-of-week adjusted linear regression on moving averages of energy consumption. These two methods are compared against state-of-the-art artificial neural networks. Energy demand predictor algorithms hav…
The 4th generation of district heating systems face a potential problem where lowered water temperatures lead to higher flow rates, which requires higher hydraulic capacity in terms of pipe and pump sizes. This increases the effect of the already existing issue of hydraulic bottlenecks, causing peripheral units (customers) to experience reduced flow rates. A coordinating control strategy is presented in this work aimed at reducing the effect of such bottlenecks on the comfort of customers. This is done by distributing the flow deficit over many units rather than a few. Previous works mainly fo…
Although electric power networks and district heating networks are physically coupled, they are not operated in a coordinated manner. With increasing penetration of renewable energy sources, a coordinated market-based operation of the two networks can yield significant advantages, as reduced need for grid reinforcements, by optimizing the power flows in the coupled systems. Transactive control has been developed as a promising approach based on market and control mechanisms to coordinate supply and demand in energy systems, which when applied to power systems is being referred to as transactiv…
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究所）；Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室）；Melentiev Energy Systems Institute of Siberian Branch of Russian Academy of Sciences（俄罗斯科学院西伯利亚分院梅连季耶夫能源系统研究所）
Combined heat and power dispatch promotes interactions and synergies between electric power systems and district heating systems. However, nonlinear and nonconvex heating flow imposes significant challenges on finding qualified solutions efficiently. Most existing methods rely on constant flow assumptions to derive a linear heating flow model, sacrificing optimality for computational simplicity. This paper proposes a novel convex combined heat and power dispatch model based on model simplification and constraint relaxation, which improves solution quality and avoids assumptions on operating re…
This paper presents a workflow to evaluate the macro scale thermal fairness for producers in a district heating network during the conceptual design phase of such a network. It uses the workflow to evaluate two types of proposed topologies for a future network to be constructed in Europe. The workflow is also novel in the sense that it has been demonstrated in the paper that within the implementation, the load balancing step can be solved readily by a quantum computing system (in particular, a quantum annealer from DWave). The concept of fairness is also addressed in the workflow and the resul…
With the increased amount of volatile renewable energy sources connected to the electricity grid, and the phase-out of fossil fuel based power plants, there is an increased need for frequency regulation. On the demand side, frequency regulation services can be offered by buildings or districts that are equipped with electric heating or cooling systems, by exploiting their thermal inertia. Existing approaches for tapping into this potential typically rely on dynamic building models, which in practice can be challenging to obtain and maintain. As a result, practical implementations of such syste…
Heating, ventilation, and air conditioning (HVAC) systems are well proved to be capable to provide operating reserve for power systems. As a type of large-capacity and energy-efficient HVAC system (up to 100 MW), district cooling system (DCS) is emerging in modern cities and has huge potential to be regulated as a flexible load. However, strategically controlling a DCS to provide flexibility is challenging, because one DCS services multiple buildings with complex thermal dynamics and uncertain cooling demands. Improper control may lead to significant thermal discomfort and even deteriorate the…
As Internet of Things (IoT) technologies enable greater communication between energy assets in smart cities, the operational coordination of various energy networks in a city or district becomes more viable. Suitable tools are needed that can harness advanced control and machine learning techniques to achieve environmental, economic and resilience objectives. In this paper, an energy management tool is presented that can offer optimal control, scheduling, forecasting and coordination services to energy assets across a district, enabling optimal decisions under user-defined objectives. The tool…
The increasing urban population imposes a substantial and growing burden on the supporting infrastructure, such as electricity, water, heating, natural gas, road transportation, etc. This paper presents a Hetero-functional graph theory (HFGT) based modeling framework for these integrated infrastructures followed by an analysis of network robustness. The supporting infrastructures along with the infrastructure repair facilities are considered. In contrast to conventional graph representations, a weighted HFGT model is used to capture the system processes and mutual dependencies among resources.…
