基于多Agent深度强化学习的多能源工业园区能源管理
Energy Management Based on Multi-Agent Deep Reinforcement Learning for A Multi-Energy Industrial Park
摘要：
由于工业能源消耗量大，工业生产在可再生能源接入和供电方面给电网带来了巨大负担。由于多能源的耦合以及可再生能源和需求的不确定性，集中式方法需要大量的计算和协调开销。因此，本文提出了一个多能源管理框架，实现分散执行和集中培训的工业园区。能源管理问题被制定为一个部分可观察的马尔可夫决策过程，这是棘手的动态规划，由于缺乏先验知识的基础随机过程。目标是在确保用户需求的同时尽量减少长期能源成本。为了解决这个问题并提高计算速度，提出了一种新型的多智能体深度强化学习算法，该算法包含以下要点：反事实基线，用于促进贡献智能体学习更好的策略;软行动者-批评者，用于提高鲁棒性和探索最优解。利用拉格朗日乘子法设计了一种新的奖励机制，以保证能量存储的容量约束。此外，考虑到Agent数量的增加会导致观测空间过大而导致性能下降的问题，引入了注意力机制，增强了策略的稳定性，使Agent能够集中注意力于重要的能量相关信息，提高了软Actor-Critic的探索效率.基于实际数据的算例验证了该算法的可扩展性，表明工业园区在不同需求下均能实现能源成本最小化。
英文摘要：
Owing to large industrial energy consumption, industrial production has brought a huge burden to the grid in terms of renewable energy access and power supply. Due to the coupling of multiple energy sources and the uncertainty of renewable energy and demand, centralized methods require large calculation and coordination overhead. Thus, this paper proposes a multi-energy management framework achieved by decentralized execution and centralized training for an industrial park. The energy management problem is formulated as a partially-observable Markov decision process, which is intractable by dynamic programming due to the lack of the prior knowledge of the underlying stochastic process. The objective is to minimize long-term energy costs while ensuring the demand of users. To solve this issue and improve the calculation speed, a novel multi-agent deep reinforcement learning algorithm is proposed, which contains the following key points: counterfactual baseline for facilitating contributing agents to learn better policies, soft actor-critic for improving robustness and exploring optimal solutions. A novel reward is designed by Lagrange multiplier method to ensure the capacity constraints of energy storage. In addition, considering that the increase in the number of agents leads to performance degradation due to large observation spaces, an attention mechanism is introduced to enhance the stability of policy and enable agents to focus on important energy-related information, which improves the exploration efficiency of soft actor-critic. Numerical results based on actual data verify the performance of the proposed algorithm with high scalability, indicating that the industrial park can minimize energy costs under different demands.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
