混合强化学习用于优化真实供水管网中的泵可持续性
Hybrid Reinforcement Learning for Optimizing Pump Sustainability in Real-World Water Distribution Networks
AI总结：
本文提出混合强化学习方法，结合历史数据与强化学习优化供水管网泵调度，降低能耗和成本，增强可解释性和鲁棒性，提升可持续性与运行效率。
摘要：
本文针对泵调度优化问题，旨在增强真实世界供水管网（WDNs）的实时控制。我们的主要目标是在遵守物理运行约束的同时，降低能耗和运行成本。传统优化技术，如基于进化的算法和遗传算法，往往因缺乏收敛保证而表现不佳。相反，强化学习（RL）因其对不确定性的适应性和更短的推理时间而脱颖而出，能够实现实时响应。然而，强化学习的有效实施依赖于为供水管网构建精确的仿真模型，而先前的应用受到仿真训练数据误差的限制。这些误差可能导致强化学习智能体学习到误导性的模式和动作，并推荐次优的运行策略。为克服这些挑战，我们提出了一种改进的“混合强化学习”方法。该方法整合了强化学习的优势，同时将其锚定在历史数据中，以历史数据为基线，逐步引入最优控制建议。通过利用运行数据作为智能体动作的基础，我们增强了智能体动作的可解释性，促进了更稳健的建议，并最小化误差。我们的研究结果表明，混合强化学习智能体能够显著提高可持续性、运行效率，并动态适应真实世界供水管网中出现的新场景。
英文摘要：
This article addresses the pump-scheduling optimization problem to enhance real-time control of real-world water distribution networks (WDNs). Our primary objectives are to adhere to physical operational constraints while reducing energy consumption and operational costs. Traditional optimization techniques, such as evolution-based and genetic algorithms, often fall short due to their lack of convergence guarantees. Conversely, reinforcement learning (RL) stands out for its adaptability to uncertainties and reduced inference time, enabling real-time responsiveness. However, the effective implementation of RL is contingent on building accurate simulation models for WDNs, and prior applications have been limited by errors in simulation training data. These errors can potentially cause the RL agent to learn misleading patterns and actions and recommend suboptimal operational strategies. To overcome these challenges, we present an improved "hybrid RL" methodology. This method integrates the benefits of RL while anchoring it in historical data, which serves as a baseline to incrementally introduce optimal control recommendations. By leveraging operational data as a foundation for the agent's actions, we enhance the explainability of the agent's actions, foster more robust recommendations, and minimize error. Our findings demonstrate that the hybrid RL agent can significantly improve sustainability, operational efficiency, and dynamically adapt to emerging scenarios in real-world WDNs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
