基于代理模型强化学习的地质CO₂封存压力管理策略优化
Optimization of Pressure Management Strategies for Geological CO2 Sequestration Using Surrogate Model-based Reinforcement Learning
AI总结:
针对地质CO₂封存压力管理策略优化计算成本过高的问题,提出基于“嵌入到控制”代理模型的强化学习方法,在满足约束的同时最大化经济效益,经咸水层封存模拟验证效果显著。
摘要:
将温室气体注入深层地下储层进行永久封存时,若注入引发的应力超过临界阈值,可能意外导致断层重新激活、盖层破裂及温室气体泄漏。在注入过程的不同阶段提取原有流体的操作被称为压力管理,可缓解相关风险并降低环境影响。然而,由于需要进行函数评估,确定最优压力管理策略通常需要数千次全阶模拟,导致计算成本过高,难以实施。\n本文提出了一种新颖的基于代理模型的强化学习方法,可高效制定地质CO₂封存的最优压力管理策略。该方法包含两个步骤:首先,通过“嵌入到控制”方法构建代理模型,该模型采用编码器-过渡层-解码器结构学习潜在动力学;借助该代理模型,利用强化学习寻找在满足各类控制约束的同时最大化经济效益的最优策略。强化学习智能体接收潜在状态空间表示和为CO₂封存定制的即时奖励,并选择受预设工程约束的实时控制动作,以最大化长期累积奖励。\n为验证该方法的有效性,将该框架应用于一个CO₂注入咸水层的组分模拟模型。结果表明,本文提出的基于代理模型的强化学习方法显著优化了CO₂封存策略,与基准场景相比带来了可观的经济收益。
英文摘要:
Injecting greenhouse gas into deep underground reservoirs for permanent storage can inadvertently lead to fault reactivation, caprock fracturing and greenhouse gas leakage when the injection-induced stress exceeds the critical threshold. Extraction of pre-existing fluids at various stages of injection process, referred as pressure management, can mitigate associated risks and lessen environmental impact. However, identifying optimal pressure management strategies typically requires thousands of full-order simulations due to the need for function evaluations, making the process computationally prohibitive. This paper introduces a novel surrogate model-based reinforcement learning method for devising optimal pressure management strategies for geological CO2 sequestration efficiently. Our approach comprises two steps. Firstly, a surrogate model is developed through the embed to control method, which employs an encoder-transition-decoder structure to learn latent dynamics. Leveraging this proxy model, reinforcement learning is utilized to find an optimal strategy that maximizes economic benefits while satisfying various control constraints. The reinforcement learning agent receives the latent state space representation and immediate reward tailored for CO2 sequestration and choose real-time controls which are subject to predefined engineering constraints in order to maximize the long-term cumulative rewards. To demonstrate its effectiveness, this framework is applied to a compositional simulation model where CO2 is injected into saline aquifer. The results reveal that our surrogate model-based reinforcement learning approach significantly optimizes CO2 sequestration strategies, leading to notable economic gains compared to baseline scenarios.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。