Closed-loop management of geological CO2 storage requires control policies that adapt to uncertain reservoir behavior while relying on observations that are realistically available during operation. This work formulates CO2 injection and brine-production control as a partially observable sequential decision problem and studies deployable deep reinforcement-learning controllers trained with high-fidelity reservoir simulation. We first compare privileged-state, well-only, history-conditioned, masking-curriculum, and asymmetric teacher-student model-free policies in order to quantify the value of temporal well-response information and training-time privileged simulator states. We then evaluate a latent model-based adaptation pipeline that reuses nominal latent dynamics and retunes controllers under known injector failure, leakage-induced dynamics and reward shift, and compartmentalized reservoir connectivity. The results show that history-conditioned policies recover nearly all of the privileged-state performance while using only deployable well-level information, and that latent model-based retuning outperforms direct model-free retuning under the same scenario-specific real-simulator budget in the abnormal operating cases. The proposed framework therefore provides a simulator-budget-aware alternative to repeated online history matching and re-optimization for closed-loop CO2 storage control.
The Ohio State University（俄亥俄州立大学）；University of Utah（犹他大学）；Southern University of Science and Technology（南方科技大学）
Variance-reduced algorithms, although achieve great theoretical performance, can run slowly in practice due to the periodic gradient estimation with a large batch of data. Batch-size adaptation thus arises as a promising approach to accelerate such algorithms. However, existing schemes either apply prescribed batch-size adaption rule or exploit the information along optimization path via additional backtracking and condition verification steps. In this paper, we propose a novel scheme, which eliminates backtracking line search but still exploits the information along optimization path by adapt…
Moscow Institute of Physics and Technology（莫斯科物理技术学院）；Artificial Intelligence Research Institute（人工智能研究所）
Model-based reinforcement learning (MBRL) allows solving complex tasks in a sample-efficient manner. However, no information is reused between the tasks. In this work, we propose a meta-learned addressing model called RAMa that provides training samples for the MBRL agent taken from continuously growing task-agnostic storage. The model is trained to maximize the expected agent's performance by selecting promising trajectories solving prior tasks from the storage. We show that such retrospective exploration can accelerate the learning process of the MBRL agent by better informing learned dynami…
Nankai University（南开大学）；Nanjing University of Science and Technology（南京理工大学）；Shanghai AI Laboratory（上海人工智能实验室）；Megvii Technology（旷视科技）
New knowledge originates from the old. The various types of elements, deposited in the training history, are a large amount of wealth for improving learning deep models. In this survey, we comprehensively review and summarize the topic--``Historical Learning: Learning Models with Learning History'', which learns better neural models with the help of their learning history during its optimization, from three detailed aspects: Historical Type (what), Functional Part (where) and Storage Form (how). To our best knowledge, it is the first survey that systematically studies the methodologies which m…
Generalized in-hand manipulation has long been an unsolved challenge of robotics. As a small step towards this grand goal, we demonstrate how to design and learn a simple adaptive controller to achieve in-hand object rotation using only fingertips. The controller is trained entirely in simulation on only cylindrical objects, which then - without any fine-tuning - can be directly deployed to a real robot hand to rotate dozens of objects with diverse sizes, shapes, and weights over the z-axis. This is achieved via rapid online adaptation of the controller to the object properties using only prop…
Accelerated training of deep learning surrogate models for surface displacement and flow, with application to MCMC-based history matching of CO2 storage operations
TFMAdapter: Lightweight Instance-Level Adaptation of Foundation Models for Forecasting with Covariates
Indian Institute of Technology Bombay Centre for Machine Intelligence（印度理工学院班加罗尔中心人工智能与数据科学中心）；Indian Institute of Technology Bombay Department of Computer Science（印度理工学院班加罗尔计算机科学与工程系）
Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）；Total E&P Research and Technology（道达尔勘探与生产研究和技术公司）；Energy & Environmental Research Center, University of North Dakota（北达科他大学能源与环境研究中心）
Fast assimilation of monitoring data to update forecasts of pressure buildup and carbon dioxide (CO2) plume migration under geologic uncertainties is a challenging problem in geologic carbon storage. The high computational cost of data assimilation with a high-dimensional parameter space impedes fast decision-making for commercial-scale reservoir management. We propose to leverage physical understandings of porous medium flow behavior with deep learning techniques to develop a fast history matching-reservoir response forecasting workflow. Applying an Ensemble Smoother Multiple Data Assimilatio…
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）；TOTAL Research and Development（道达尔研发中心）
Reservoir simulation and adaptation (also known as history matching) are typically considered as separate problems. While a set of models are aimed at the solution of the forward simulation problem assuming all initial geological parameters are known, the other set of models adjust geological parameters under the fixed forward simulation model to fit production data. This results in many difficulties for both reservoir engineers and developers of new efficient computation schemes. We present a unified approach to reservoir simulation and adaptation problems. A single neural network model allow…
Rapid Embodiment Adaptation for Quadrupedal Locomotion
UC San Diego（加州大学圣迭戈分校）；Stanford University（斯坦福大学）；TU Darmstadt（达姆施塔特工业大学）；Sudo AI GmbH（Sudo AI 有限公司）；German Research Center for AI (DFKI)（德国人工智能研究中心）；Robotics Institute Germany（德国机器人研究所）
