Efficient Uncertainty Quantification for Dynamic Subsurface Flow with Surrogate by Theory-guided Neural Network
College of Engineering, Peking University（北京大学工学院）；School of Environmental Science and Engineering, Southern University of Science and Technology（南方科技大学环境科学与工程学院）；Peng Cheng Laboratory（鹏城实验室）
Subsurface flow problems usually involve some degree of uncertainty. Consequently, uncertainty quantification is commonly necessary for subsurface flow prediction. In this work, we propose a methodology for efficient uncertainty quantification for dynamic subsurface flow with a surrogate constructed by the Theory-guided Neural Network (TgNN). The TgNN here is specially designed for problems with stochastic parameters. In the TgNN, stochastic parameters, time and location comprise the input of the neural network, while the quantity of interest is the output. The neural network is trained with available simulation data, while being simultaneously guided by theory (e.g., the governing equation, boundary conditions, initial conditions, etc.) of the underlying problem. The trained neural network can predict solutions of subsurface flow problems with new stochastic parameters. With the TgNN surrogate, the Monte Carlo (MC) method can be efficiently implemented for uncertainty quantification. The proposed methodology is evaluated with two-dimensional dynamic saturated flow problems in porous medium. Numerical results show that the TgNN based surrogate can significantly improve the efficiency of uncertainty quantification tasks compared with simulation based implementation. Further investigations regarding stochastic fields with smaller correlation length, larger variance, changing boundary values and out-of-distribution variances are performed, and satisfactory results are obtained.
SURGIN: SURrogate-guided Generative INversion for subsurface multiphase flow with quantified uncertainty
Department of Geotechnical Engineering, College of Civil Engineering, Tongji University（同济大学土木工程学院地质工程系）；Key Laboratory of Geotechnical and Underground Engineering of Ministry of Education, Tongji University（教育部同济大学地质与地下工程重点实验室）；Physical Science and Engineering Division, King Abdullah University of Science and Technology (KAUST)（卡布斯科学与技术大学（KAUST）物理科学与工程系）；State Key Laboratory of Marine Geology, Tongji University（同济大学海洋地质国家重点实验室）；Tencent SSV（腾讯SSV）；Tencent SD（腾讯SD）
AutoSurrogate: An LLM-Driven Multi-Agent Framework for Autonomous Construction of Deep Learning Surrogate Models in Subsurface Flow
School of Physics and Astronomy, The University of Edinburgh（物理与天文学学院，爱丁堡大学）；Institute of GeoEnergy Engineering, School of Energy, Geoscience, Infrastructure and Society, Heriot-Watt University（地球能源工程研究所，能源、地质科学、基础设施与社会学院，赫瑞斯泰大学）；Subsurface Energy Transition and Innovation Centre, Heriot-Watt University（地下能源转型与创新中心，赫瑞斯泰大学）
Peng Cheng Laboratory（鹏城实验室）；Southern University of Science and Technology（南方科技大学）；Peking University（北京大学）
We build surrogate models for dynamic 3D subsurface single-phase flow problems with multiple vertical producing wells. The surrogate model provides efficient pressure estimation of the entire formation at any timestep given a stochastic permeability field, arbitrary well locations and penetration lengths, and a timestep matrix as inputs. The well production rate or bottom hole pressure can then be determined based on Peaceman's formula. The original surrogate modeling task is transformed into an image-to-image regression problem using a convolutional encoder-decoder neural network architecture…
We are interested in the development of surrogate models for uncertainty quantification and propagation in problems governed by stochastic PDEs using a deep convolutional encoder-decoder network in a similar fashion to approaches considered in deep learning for image-to-image regression tasks. Since normal neural networks are data intensive and cannot provide predictive uncertainty, we propose a Bayesian approach to convolutional neural nets. A recently introduced variational gradient descent algorithm based on Stein's method is scaled to deep convolutional networks to perform approximate Baye…
We propose a novel \textit{capsule} based deep encoder-decoder model for surrogate modeling and uncertainty quantification of systems in mechanics from sparse data. The proposed framework is developed by adapting Capsule Network (CapsNet) architecture into image-to-image regression encoder-decoder network. Specifically, the aim is to exploit the benefits of CapsNet over convolution neural network (CNN) $-$ retaining pose and position information related to an entity to name a few. The performance of proposed approach is illustrated by solving an elliptic stochastic partial differential equatio…
State-of-the-art computer codes for simulating real physical systems are often characterized by a vast number of input parameters. Performing uncertainty quantification (UQ) tasks with Monte Carlo (MC) methods is almost always infeasible because of the need to perform hundreds of thousands or even millions of forward model evaluations in order to obtain convergent statistics. One, thus, tries to construct a cheap-to-evaluate surrogate model to replace the forward model solver. For systems with large numbers of input parameters, one has to deal with the curse of dimensionality - the exponential…
For computational efficiency, surrogate models have been used to emulate mathematical simulators for physical or biological processes. High-speed simulation is crucial for conducting uncertainty quantification (UQ) when the simulation is repeated over many randomly sampled input points (aka, the Monte Carlo method). In some cases, UQ is only feasible with a surrogate model. Recently, Deep Neural Network (DNN) surrogate models have gained popularity for their hard-to-match emulation accuracy. However, it is well-known that DNN is prone to errors when input data are perturbed in particular ways,…
University of Notre Dame（圣母大学）；Technical University of Munich（慕尼黑工业大学）；University of Pennsylvania（宾夕法尼亚大学）
Surrogate modeling and uncertainty quantification tasks for PDE systems are most often considered as supervised learning problems where input and output data pairs are used for training. The construction of such emulators is by definition a small data problem which poses challenges to deep learning approaches that have been developed to operate in the big data regime. Even in cases where such models have been shown to have good predictive capability in high dimensions, they fail to address constraints in the data implied by the PDE model. This paper provides a methodology that incorporates the…
Bangladesh University of Engineering and Technology（孟加拉国工程技术大学）；Jashore University of Science and Technology（杰索尔科技大学）；Indian Institute of Technology Delhi（印度理工学院德里分校）；University of Bristol（布里斯托大学）；Missouri University of Science and Technology（密苏里科技大学）
Industry 4.0 targets the conversion of the traditional industries into intelligent ones through technological revolution. This revolution is only possible through innovation, optimization, interconnection, and rapid decision-making capability. Numerical models are believed to be the key components of Industry 4.0, facilitating quick decision-making through simulations instead of costly experiments. However, numerical investigation of precise, high-fidelity models for optimization or decision-making is usually time-consuming and computationally expensive. In such instances, data-driven surrogat…
TU Dresden（德累斯顿工业大学）；University of Bristol（布里斯托大学）
Explainability of black-box machine learning models is crucial, in particular when deployed in critical applications such as medicine or autonomous cars. Existing approaches produce explanations for the predictions of models, however, how to assess the quality and reliability of such explanations remains an open question. In this paper we take a step further in order to provide the practitioner with tools to judge the trustworthiness of an explanation. To this end, we produce estimates of the uncertainty of a given explanation by measuring the ordinal consensus amongst a set of diverse bootstr…
Indian Institute of Technology Guwahati（古瓦哈提印度理工学院）；Indian Institute of Technology Delhi（德里印度理工学院）
We propose a novel deep learning based surrogate model for solving high-dimensional uncertainty quantification and uncertainty propagation problems. The proposed deep learning architecture is developed by integrating the well-known U-net architecture with the Gaussian Gated Linear Network (GGLN) and referred to as the Gated Linear Network induced U-net or GLU-net. The proposed GLU-net treats the uncertainty propagation problem as an image to image regression and hence, is extremely data efficient. Additionally, it also provides estimates of the predictive uncertainty. The network architecture …
National Cheng Kung University（国立成功大学）；Wells Fargo（富国银行）；Georgia Institute of Technology（佐治亚理工学院）
Global optimization of expensive functions has important applications in physical and computer experiments. It is a challenging problem to develop efficient optimization scheme, because each function evaluation can be costly and the derivative information of the function is often not available. We propose a novel global optimization framework using adaptive Radial Basis Functions (RBF) based surrogate model via uncertainty quantification. The framework consists of two iteration steps. It first employs an RBF-based Bayesian surrogate model to approximate the true function, where the parameters …
We present a hybrid sampling-surrogate approach for reducing the computational expense of uncertainty quantification in nonlinear dynamical systems. Our motivation is to enable rapid uncertainty quantification in complex mechanical systems such as automotive propulsion systems. Our approach is to build upon ideas from multifidelity uncertainty quantification to leverage the benefits of both sampling and surrogate modeling, while mitigating their downsides. In particular, the surrogate model is selected to exploit problem structure, such as smoothness, and offers a highly correlated information…
Uncertainty Quantification for Reduced-Order Surrogate Models Applied to Cloud Microphysics
Department of Applied & Computational Mathematics（应用与计算数学系）；Yale University（耶鲁大学）；Atmospheric, Earth, & Energy Division（大气、地球与能源部门）；Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）
InstroTek Inc.（InstroTek公司）；North Carolina State University（北卡罗来纳州立大学）
Full Wave Inversion (FWI) imaging scheme has many applications in engineering, geoscience and medical sciences. In this paper, a surrogate deep learning FWI approach is presented to quantify properties of materials using stress waves. Such inverse problems, in general, are ill-posed and nonconvex, especially in cases where the solutions exhibit shocks, heterogeneity, discontinuities, or large gradients. The proposed approach is proven efficient to obtain global minima responses in these cases. This approach is trained based on random sampled set of material properties and sampled trials around…
Uncertainty Quantification for AI-Driven Crash Simulation Surrogates: A Comparative Study of Monte Carlo Dropout and Deep Ensemble on Open-Source Bumper Beam Benchmark
Uncertainty estimations are presented of the response of a multiscale in-stent restenosis model, as obtained by both non-intrusive and semi-intrusive uncertainty quantification. The in-stent restenosis model is a fully coupled multiscale simulation of post-stenting tissue growth, in which the most costly submodel is the blood flow simulation. Surrogate modeling for non-intrusive uncertainty quantification takes the whole model as a black-box and maps directly from the three uncertain inputs to the quantity of interest, the neointimal area. The corresponding uncertain estimates matched the resu…
We consider the point evaluation of the solution to interface problems with geometric uncertainties, where the uncertainty in the obstacle is described by a high-dimensional parameter $\boldsymbol{y}\in[-1,1]^d$, $d\in\mathbb{N}$. We focus in particular on an elliptic interface problem and a Helmholtz transmission problem. Point values of the solution in the physical domain depend in general non-smoothly on the high-dimensional parameter, posing a challenge when one is interested in building surrogates. Indeed, high-order methods show poor convergence rates, while methods which are able to tra…
Uncertainty quantification of two-phase flow in porous media via coupled-TgNN surrogate model
China University of Petroleum (East China)（中国石油大学（华东））；Southern University of Science and Technology（南方科技大学）；Peng Cheng Laboratory（鹏城实验室）；Peking University（北京大学）
Direct Interval Propagation Methods using Neural-Network Surrogates for Uncertainty Quantification in Physical Systems Surrogate Model
LMSD, Department of Mechanical Engineering, KU Leuven（KU莱顿机械工程系LMSD）；RAM, Department of Mechanical Engineering, KU Leuven（KU莱顿机械工程系RAM）；DistriNet, Department of Computer Science, KU Leuven（KU莱顿计算机科学系DistriNet）；Visual Artificial Intelligence Laboratory, Oxford Brookes University（奥克斯福德布鲁克斯大学视觉人工智能实验室）
Seismic imaging is an ill-posed inverse problem that is challenged by noisy data and modeling inaccuracies -- due to errors in the background squared-slowness model. Uncertainty quantification is essential for determining how variability in the background models affects seismic imaging. Due to the costs associated with the forward Born modeling operator as well as the high dimensionality of seismic images, quantification of uncertainty is computationally expensive. As such, the main contribution of this work is a survey-specific Fourier neural operator surrogate to velocity continuation that m…
Surrogate Modelling of Proton Dose with Monte Carlo Dropout Uncertainty Quantification
Institute for Mathematical Innovation University of Bath, Bath, UK（数学创新研究所 英国巴斯大学）；Department of Mathematical Sciences University of Bath, Bath, UK（数学科学系 英国巴斯大学）
University of Amsterdam（阿姆斯特丹大学）；ITMO University（圣光机大学）
In-Stent Restenosis is a recurrence of coronary artery narrowing due to vascular injury caused by balloon dilation and stent placement. It may lead to the relapse of angina symptoms or to an acute coronary syndrome. An uncertainty quantification of a model for In-Stent Restenosis with four uncertain parameters (endothelium regeneration time, the threshold strain for smooth muscle cells bond breaking, blood flow velocity and the percentage of fenestration in the internal elastic lamina) is presented. Two quantities of interest were studied, namely the average cross-sectional area and the maximu…
Peking University（北京大学）；Southern University of Science and Technology（南方科技大学）
A Theory-guided Auto-Encoder (TgAE) framework is proposed for surrogate construction and is further used for uncertainty quantification and inverse modeling tasks. The framework is built based on the Auto-Encoder (or Encoder-Decoder) architecture of convolutional neural network (CNN) via a theory-guided training process. In order to achieve the theory-guided training, the governing equations of the studied problems can be discretized and the finite difference scheme of the equations can be embedded into the training of CNN. The residual of the discretized governing equations as well as the dat…
Micro-Macro Tensor Neural Surrogates for Uncertainty Quantification in Collisional Plasma
University of Ferrara（费拉拉大学）；Italian Ministry of University and Research（意大利大学与研究部）；Royal Society（皇家学会）；Italian National Institute of High Mathematics（意大利高等数学研究院）
National University of Defense Technology（国防科技大学）；Chinese Academy of Military Science（中国人民解放军军事科学院）
The surrogate model-based uncertainty quantification method has drawn much attention in many engineering fields. Polynomial chaos expansion (PCE) and deep learning (DL) are powerful methods for building a surrogate model. However, PCE needs to increase the expansion order to improve the accuracy of the surrogate model, which causes more labeled data to solve the expansion coefficients, and DL also requires a lot of labeled data to train the deep neural network (DNN). First of all, this paper proposes the adaptive arbitrary polynomial chaos (aPC) and proves two properties about the adaptive exp…
Structure and asymptotic preserving deep neural surrogates for uncertainty quantification in multiscale kinetic equations
University of Ferrara（费拉拉大学）；Italian Ministry of University and Research（意大利大学与研究部）；Royal Society（皇家学会）；European Union（欧盟）；ICSC – Centro Nazionale di Ricerca in High Performance Computing, Big Data and Quantum Computing（ICSC – 高性能计算、大数据和量子计算研究中心）；Italian National Institute of High Mathematics（意大利国家高等数学研究所）
Uncertainty Quantification for Machine Learning-Based Prediction: A Polynomial Chaos Expansion Approach for Joint Model and Input Uncertainty Propagation
School of Mechanical Engineering（机械工程学院）；Purdue University（普渡大学）
University of Notre Dame（圣母大学）；Nanjing University（南京大学）
Surrogate strategies are used widely for uncertainty quantification of groundwater models in order to improve computational efficiency. However, their application to dynamic multiphase flow problems is hindered by the curse of dimensionality, the saturation discontinuity due to capillarity effects, and the time-dependence of the multi-output responses. In this paper, we propose a deep convolutional encoder-decoder neural network methodology to tackle these issues. The surrogate modeling task is transformed to an image-to-image regression strategy. This approach extracts high-level coarse featu…
AI enhanced data assimilation and uncertainty quantification applied to Geological Carbon Storage
Faculty of Civil Engineering；Geosciences, TU Delft, Stevinweg 1, 2628 CN Delft, Netherlands；Centrum Wiskunde \& Informatica, Science Park 123, 1098 XG Amsterdam, Netherlands；Mathematical Institute, Utrecht University, Utrecht, Netherlands；Department of Earth Science；Engineering, Imperial College London, London, United Kingdom；Department of Energy Resources Engineering, Stanford University, CA, USA
University of New South Wales（新南威尔士大学）；The University of Sydney（悉尼大学）；Indian Institute of Technology Guwahati（印度理工学院古瓦哈提分校）；SRM Institute of Science and Technology（SRM科学技术学院）；Indian Institute of Technology Delhi（印度理工学院德里分校）
Due to the need for robust uncertainty quantification, Bayesian neural learning has gained attention in the era of deep learning and big data. Markov Chain Monte-Carlo (MCMC) methods typically implement Bayesian inference which faces several challenges given a large number of parameters, complex and multimodal posterior distributions, and computational complexity of large neural network models. Parallel tempering MCMC addresses some of these limitations given that they can sample multimodal posterior distributions and utilize high-performance computing. However, certain challenges remain given…
Graz University of Technology（格拉茨工业大学）；Graz Center for Computational Engineering（格拉茨计算工程中心）；University Medicine Essen（埃森大学医学院）；Norwegian University of Science and Technology (NTNU)（挪威科技大学）
Inhomogeneities in the aortic wall can lead to localized stress accumulations, possibly initiating dissection. In many cases, a dissection results from pathological changes such as fragmentation or loss of elastic fibers. But it has been shown that even the healthy aortic wall has an inherent heterogeneous microstructure. Some parts of the aorta are particularly susceptible to the development of inhomogeneities due to pathological changes, however, the distribution in the aortic wall and the spatial extent, such as size, shape, and type, are difficult to predict. Motivated by this observation,…
BMW Group（宝马集团）；Munich Data Science Institute（慕尼黑数据科学研究所）；Munich Center for Machine Learning（慕尼黑机器学习中心）
Efficient surrogate modelling is a key requirement for uncertainty quantification in data-driven scenarios. In this work, a novel approach of using Sparse Random Features for surrogate modelling in combination with self-supervised dimensionality reduction is described. The method is compared to other methods on synthetic and real data obtained from crashworthiness analyses. The results show a superiority of the here described approach over state of the art surrogate modelling techniques, Polynomial Chaos Expansions and Neural Networks.
University of New South Wales（新南威尔士大学）；University of Sydney（悉尼大学）；SRM Institute of Science and Technology（SRM科学技术研究所）
The complex and computationally expensive nature of landscape evolution models pose significant challenges in the inference and optimisation of unknown parameters. Bayesian inference provides a methodology for estimation and uncertainty quantification of unknown model parameters. In our previous work, we developed parallel tempering Bayeslands as a framework for parameter estimation and uncertainty quantification for the Badlands landscape evolution model. Parallel tempering Bayeslands features high-performance computing with dozens of processing cores running in parallel to enhance computatio…
Johns Hopkins University（约翰斯·霍普金斯大学）；Technische Universität Darmstadt（达姆施塔特工业大学）；Siemens AG（西门子股份公司）
Constructing surrogate models for uncertainty quantification (UQ) on complex partial differential equations (PDEs) having inherently high-dimensional $\mathcal{O}(10^{\ge 2})$ stochastic inputs (e.g., forcing terms, boundary conditions, initial conditions) poses tremendous challenges. The curse of dimensionality can be addressed with suitable unsupervised learning techniques used as a pre-processing tool to encode inputs onto lower-dimensional subspaces while retaining its structural information and meaningful properties. In this work, we review and investigate thirteen dimension reduction met…
