AI 中文总结
针对EGS实时决策的高计算成本问题，提出扩散代理引导的强化学习框架，结合条件扩散模型与PPO，在裂缝型EGS基准上实现高效井控优化，性能优于现有方法且降低模拟依赖。
AI 中文摘要
增强型地热系统（EGS）的实时决策极具挑战性，因为长期生产周期涉及高维控制空间以及大量耗时的高保真热液模拟。强化学习为基于状态的序列控制提供了自然框架，但直接使用数值模拟器进行策略训练的计算成本很高。为解决该问题，我们提出一种扩散代理引导的强化学习框架，用于长视野EGS井控优化。将储层温度和压力场作为系统状态，注入速率选为控制动作。使用条件扩散模型构建学习到的代理环境，以预测储层温度和压力场的演化，同时用单独的奖励模型估计相应的经济回报。随后将该代理环境与近端策略优化（PPO）集成，实现高效的策略训练。在裂缝型EGS基准测试上的实验表明，扩散代理可准确复现多个控制阶段的储层状态演化。与基于直接模拟器的PPO及现有优化方法相比，所得的代理辅助PPO策略达到了具有竞争力的井控性能，同时大幅降低了对昂贵高保真模拟的依赖。这些结果证明了基于扩散的代理环境在地热井控优化的高效强化学习中的潜力。
英文摘要
Real-time decision-making for enhanced geothermal systems (EGS) is challenging because long-term production periods involve high-dimensional control spaces and a large number of time-consuming high-fidelity hydrothermal simulations. Reinforcement learning provides a natural framework for state-dependent sequential control, but direct policy training with numerical simulators is computationally expensive. To address this issue, we propose a diffusion-surrogate guided reinforcement learning framework for long-horizon EGS well-control optimization. The reservoir temperature and pressure fields are used as system states, while injection rates are selected as control actions. A learned surrogate environment is constructed using conditional diffusion models to predict the evolution of reservoir temperature and pressure fields and a separate reward model to estimate the corresponding economic return. The surrogate environment is then integrated with Proximal Policy Optimization (PPO) for efficient policy training. Experiments on a fractured EGS benchmark show that the diffusion surrogate can accurately reproduce reservoir-state evolution over multiple control stages. The resulting surrogate-assisted PPO policy achieves competitive well-control performance compared with direct simulator-based PPO and existing optimization methods, while substantially reducing the dependence on expensive high-fidelity simulations. These results demonstrate the potential of diffusion-based surrogate environments for efficient reinforcement learning in geothermal well-control optimization.
