Global optimization of expensive functions has important applications in physical and computer experiments. It is a challenging problem to develop efficient optimization scheme, because each function evaluation can be costly and the derivative information of the function is often not available. We propose a novel global optimization framework using adaptive Radial Basis Functions (RBF) based surrogate model via uncertainty quantification. The framework consists of two iteration steps. It first employs an RBF-based Bayesian surrogate model to approximate the true function, where the parameters of the RBFs can be adaptively estimated and updated each time a new point is explored. Then it utilizes a model-guided selection criterion to identify a new point from a candidate set for function evaluation. The selection criterion adopted here is a sample version of the expected improvement (EI) criterion. We conduct simulation studies with standard test functions, which show that the proposed method has some advantages, especially when the true surface is not very smooth. In addition, we also propose modified approaches to improve the search performance for identifying global optimal points and to deal with the higher dimension scenarios.
Automatically searching for optimal hyperparameter configurations is of crucial importance for applying deep learning algorithms in practice. Recently, Bayesian optimization has been proposed for optimizing hyperparameters of various machine learning algorithms. Those methods adopt probabilistic surrogate models like Gaussian processes to approximate and minimize the validation error function of hyperparameter values. However, probabilistic surrogates require accurate estimates of sufficient statistics (e.g., covariance) of the error distribution and thus need many function evaluations with a …
We propose a novel \textit{capsule} based deep encoder-decoder model for surrogate modeling and uncertainty quantification of systems in mechanics from sparse data. The proposed framework is developed by adapting Capsule Network (CapsNet) architecture into image-to-image regression encoder-decoder network. Specifically, the aim is to exploit the benefits of CapsNet over convolution neural network (CNN) $-$ retaining pose and position information related to an entity to name a few. The performance of proposed approach is illustrated by solving an elliptic stochastic partial differential equatio…
College of Engineering, Peking University（北京大学工学院）；School of Environmental Science and Engineering, Southern University of Science and Technology（南方科技大学环境科学与工程学院）；Peng Cheng Laboratory（鹏城实验室）
Subsurface flow problems usually involve some degree of uncertainty. Consequently, uncertainty quantification is commonly necessary for subsurface flow prediction. In this work, we propose a methodology for efficient uncertainty quantification for dynamic subsurface flow with a surrogate constructed by the Theory-guided Neural Network (TgNN). The TgNN here is specially designed for problems with stochastic parameters. In the TgNN, stochastic parameters, time and location comprise the input of the neural network, while the quantity of interest is the output. The neural network is trained with a…
For computational efficiency, surrogate models have been used to emulate mathematical simulators for physical or biological processes. High-speed simulation is crucial for conducting uncertainty quantification (UQ) when the simulation is repeated over many randomly sampled input points (aka, the Monte Carlo method). In some cases, UQ is only feasible with a surrogate model. Recently, Deep Neural Network (DNN) surrogate models have gained popularity for their hard-to-match emulation accuracy. However, it is well-known that DNN is prone to errors when input data are perturbed in particular ways,…
Virginia Tech（弗吉尼亚理工大学）；Kuwait University（科威特大学）；Department of Industrial and Systems Engineering, Virginia Tech（弗吉尼亚理工大学工业与系统工程系）；Department of Industrial and Management Systems Engineering, Kuwait University（科威特大学工业与管理系统工程系）
Some response surface functions in complex engineering systems are usually highly nonlinear, unformed, and expensive-to-evaluate. To tackle this challenge, Bayesian optimization, which conducts sequential design via a posterior distribution over the objective function, is a critical method used to find the global optimum of black-box functions. Kernel functions play an important role in shaping the posterior distribution of the estimated function. The widely used kernel function, e.g., radial basis function (RBF), is very vulnerable and susceptible to outliers; the existence of outliers is cau…
Department of Biostatistics, University of Oslo（奥斯陆大学生物统计系）；Helsinki Institute for Information Technology HIIT, Department of Computer Science, Aalto University（赫尔辛基信息技术研究所，阿尔托大学计算机科学系）
The computational efficiency of approximate Bayesian computation (ABC) has been improved by using surrogate models such as Gaussian processes (GP). In one such promising framework the discrepancy between the simulated and observed data is modelled with a GP which is further used to form a model-based estimator for the intractable posterior. In this article we improve this approach in several ways. We develop batch-sequential Bayesian experimental design strategies to parallellise the expensive simulations. In earlier work only sequential strategies have been used. Current surrogate-based ABC m…
Indian Institute of Technology Guwahati（古瓦哈提印度理工学院）；Indian Institute of Technology Delhi（德里印度理工学院）
We propose a novel deep learning based surrogate model for solving high-dimensional uncertainty quantification and uncertainty propagation problems. The proposed deep learning architecture is developed by integrating the well-known U-net architecture with the Gaussian Gated Linear Network (GGLN) and referred to as the Gated Linear Network induced U-net or GLU-net. The proposed GLU-net treats the uncertainty propagation problem as an image to image regression and hence, is extremely data efficient. Additionally, it also provides estimates of the predictive uncertainty. The network architecture …
TU Dresden（德累斯顿工业大学）；University of Bristol（布里斯托大学）
Explainability of black-box machine learning models is crucial, in particular when deployed in critical applications such as medicine or autonomous cars. Existing approaches produce explanations for the predictions of models, however, how to assess the quality and reliability of such explanations remains an open question. In this paper we take a step further in order to provide the practitioner with tools to judge the trustworthiness of an explanation. To this end, we produce estimates of the uncertainty of a given explanation by measuring the ordinal consensus amongst a set of diverse bootstr…
Bangladesh University of Engineering and Technology（孟加拉国工程技术大学）；Jashore University of Science and Technology（杰索尔科技大学）；Indian Institute of Technology Delhi（印度理工学院德里分校）；University of Bristol（布里斯托大学）；Missouri University of Science and Technology（密苏里科技大学）
Industry 4.0 targets the conversion of the traditional industries into intelligent ones through technological revolution. This revolution is only possible through innovation, optimization, interconnection, and rapid decision-making capability. Numerical models are believed to be the key components of Industry 4.0, facilitating quick decision-making through simulations instead of costly experiments. However, numerical investigation of precise, high-fidelity models for optimization or decision-making is usually time-consuming and computationally expensive. In such instances, data-driven surrogat…
We are interested in the development of surrogate models for uncertainty quantification and propagation in problems governed by stochastic PDEs using a deep convolutional encoder-decoder network in a similar fashion to approaches considered in deep learning for image-to-image regression tasks. Since normal neural networks are data intensive and cannot provide predictive uncertainty, we propose a Bayesian approach to convolutional neural nets. A recently introduced variational gradient descent algorithm based on Stein's method is scaled to deep convolutional networks to perform approximate Baye…
State-of-the-art computer codes for simulating real physical systems are often characterized by a vast number of input parameters. Performing uncertainty quantification (UQ) tasks with Monte Carlo (MC) methods is almost always infeasible because of the need to perform hundreds of thousands or even millions of forward model evaluations in order to obtain convergent statistics. One, thus, tries to construct a cheap-to-evaluate surrogate model to replace the forward model solver. For systems with large numbers of input parameters, one has to deal with the curse of dimensionality - the exponential…
Uncertainty Quantification for AI-Driven Crash Simulation Surrogates: A Comparative Study of Monte Carlo Dropout and Deep Ensemble on Open-Source Bumper Beam Benchmark
University of Notre Dame（圣母大学）；Technical University of Munich（慕尼黑工业大学）；University of Pennsylvania（宾夕法尼亚大学）
Surrogate modeling and uncertainty quantification tasks for PDE systems are most often considered as supervised learning problems where input and output data pairs are used for training. The construction of such emulators is by definition a small data problem which poses challenges to deep learning approaches that have been developed to operate in the big data regime. Even in cases where such models have been shown to have good predictive capability in high dimensions, they fail to address constraints in the data implied by the PDE model. This paper provides a methodology that incorporates the…
We consider the point evaluation of the solution to interface problems with geometric uncertainties, where the uncertainty in the obstacle is described by a high-dimensional parameter $\boldsymbol{y}\in[-1,1]^d$, $d\in\mathbb{N}$. We focus in particular on an elliptic interface problem and a Helmholtz transmission problem. Point values of the solution in the physical domain depend in general non-smoothly on the high-dimensional parameter, posing a challenge when one is interested in building surrogates. Indeed, high-order methods show poor convergence rates, while methods which are able to tra…
Direct Interval Propagation Methods using Neural-Network Surrogates for Uncertainty Quantification in Physical Systems Surrogate Model
LMSD, Department of Mechanical Engineering, KU Leuven（KU莱顿机械工程系LMSD）；RAM, Department of Mechanical Engineering, KU Leuven（KU莱顿机械工程系RAM）；DistriNet, Department of Computer Science, KU Leuven（KU莱顿计算机科学系DistriNet）；Visual Artificial Intelligence Laboratory, Oxford Brookes University（奥克斯福德布鲁克斯大学视觉人工智能实验室）
Uncertainty Quantification for Reduced-Order Surrogate Models Applied to Cloud Microphysics
Department of Applied & Computational Mathematics（应用与计算数学系）；Yale University（耶鲁大学）；Atmospheric, Earth, & Energy Division（大气、地球与能源部门）；Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）
Peking University（北京大学）；Southern University of Science and Technology（南方科技大学）
A Theory-guided Auto-Encoder (TgAE) framework is proposed for surrogate construction and is further used for uncertainty quantification and inverse modeling tasks. The framework is built based on the Auto-Encoder (or Encoder-Decoder) architecture of convolutional neural network (CNN) via a theory-guided training process. In order to achieve the theory-guided training, the governing equations of the studied problems can be discretized and the finite difference scheme of the equations can be embedded into the training of CNN. The residual of the discretized governing equations as well as the dat…
Surrogate Modelling of Proton Dose with Monte Carlo Dropout Uncertainty Quantification
Institute for Mathematical Innovation University of Bath, Bath, UK（数学创新研究所 英国巴斯大学）；Department of Mathematical Sciences University of Bath, Bath, UK（数学科学系 英国巴斯大学）
Seismic imaging is an ill-posed inverse problem that is challenged by noisy data and modeling inaccuracies -- due to errors in the background squared-slowness model. Uncertainty quantification is essential for determining how variability in the background models affects seismic imaging. Due to the costs associated with the forward Born modeling operator as well as the high dimensionality of seismic images, quantification of uncertainty is computationally expensive. As such, the main contribution of this work is a survey-specific Fourier neural operator surrogate to velocity continuation that m…
Polynomial chaos and Gaussian process emulation are methods for surrogate-based uncertainty quantification, and have been developed independently in their respective communities over the last 25 years. Despite tackling similar problems in the field, to our knowledge there has yet to be a critical comparison of the two approaches in the literature. We begin by providing a detailed description of polynomial chaos and Gaussian process approaches for building a surrogate model of a black-box function. The accuracy of each surrogate method is then tested and compared for two simulators used in indu…
Micro-Macro Tensor Neural Surrogates for Uncertainty Quantification in Collisional Plasma
University of Ferrara（费拉拉大学）；Italian Ministry of University and Research（意大利大学与研究部）；Royal Society（皇家学会）；Italian National Institute of High Mathematics（意大利高等数学研究院）
National University of Defense Technology（国防科技大学）；Chinese Academy of Military Science（中国人民解放军军事科学院）
The surrogate model-based uncertainty quantification method has drawn much attention in many engineering fields. Polynomial chaos expansion (PCE) and deep learning (DL) are powerful methods for building a surrogate model. However, PCE needs to increase the expansion order to improve the accuracy of the surrogate model, which causes more labeled data to solve the expansion coefficients, and DL also requires a lot of labeled data to train the deep neural network (DNN). First of all, this paper proposes the adaptive arbitrary polynomial chaos (aPC) and proves two properties about the adaptive exp…
University of Amsterdam（阿姆斯特丹大学）；ITMO University（圣光机大学）
In-Stent Restenosis is a recurrence of coronary artery narrowing due to vascular injury caused by balloon dilation and stent placement. It may lead to the relapse of angina symptoms or to an acute coronary syndrome. An uncertainty quantification of a model for In-Stent Restenosis with four uncertain parameters (endothelium regeneration time, the threshold strain for smooth muscle cells bond breaking, blood flow velocity and the percentage of fenestration in the internal elastic lamina) is presented. Two quantities of interest were studied, namely the average cross-sectional area and the maximu…
Uncertainty quantification of two-phase flow in porous media via coupled-TgNN surrogate model
China University of Petroleum (East China)（中国石油大学（华东））；Southern University of Science and Technology（南方科技大学）；Peng Cheng Laboratory（鹏城实验室）；Peking University（北京大学）
Peng Cheng Laboratory（鹏城实验室）；Southern University of Science and Technology（南方科技大学）；Peking University（北京大学）
We build surrogate models for dynamic 3D subsurface single-phase flow problems with multiple vertical producing wells. The surrogate model provides efficient pressure estimation of the entire formation at any timestep given a stochastic permeability field, arbitrary well locations and penetration lengths, and a timestep matrix as inputs. The well production rate or bottom hole pressure can then be determined based on Peaceman's formula. The original surrogate modeling task is transformed into an image-to-image regression problem using a convolutional encoder-decoder neural network architecture…
University of Notre Dame（圣母大学）；Nanjing University（南京大学）
Surrogate strategies are used widely for uncertainty quantification of groundwater models in order to improve computational efficiency. However, their application to dynamic multiphase flow problems is hindered by the curse of dimensionality, the saturation discontinuity due to capillarity effects, and the time-dependence of the multi-output responses. In this paper, we propose a deep convolutional encoder-decoder neural network methodology to tackle these issues. The surrogate modeling task is transformed to an image-to-image regression strategy. This approach extracts high-level coarse featu…
We propose a method for training a deterministic deep model that can find and reject out of distribution data points at test time with a single forward pass. Our approach, deterministic uncertainty quantification (DUQ), builds upon ideas of RBF networks. We scale training in these with a novel loss function and centroid updating scheme and match the accuracy of softmax models. By enforcing detectability of changes in the input using a gradient penalty, we are able to reliably detect out of distribution data. Our uncertainty quantification scales well to large datasets, and using a single model…
A Bayesian latent Gaussian process framework for aerodynamic uncertainty quantification
Aerospace Engineering, The Pennsylvania State University, University Park, PA 16802（航空工程，宾夕法尼亚州立大学，University Park分校，PA 16802）；Aerospace Engineering and the Institute of Computational and Data Science (ICDS), The Pennsylvania State University, University Park, PA 16802（航空工程和计算与数据科学研究所（ICDS），宾夕法尼亚州立大学，University Park分校，PA 16802）
Structure and asymptotic preserving deep neural surrogates for uncertainty quantification in multiscale kinetic equations
University of Ferrara（费拉拉大学）；Italian Ministry of University and Research（意大利大学与研究部）；Royal Society（皇家学会）；European Union（欧盟）；ICSC – Centro Nazionale di Ricerca in High Performance Computing, Big Data and Quantum Computing（ICSC – 高性能计算、大数据和量子计算研究中心）；Italian National Institute of High Mathematics（意大利国家高等数学研究所）
Uncertainty Quantification for Machine Learning-Based Prediction: A Polynomial Chaos Expansion Approach for Joint Model and Input Uncertainty Propagation
School of Mechanical Engineering（机械工程学院）；Purdue University（普渡大学）
Global optimization problems whose objective function is expensive to evaluate can be solved effectively by recursively fitting a surrogate function to function samples and minimizing an acquisition function to generate new samples. The acquisition step trades off between seeking for a new optimization vector where the surrogate is minimum (exploitation of the surrogate) and looking for regions of the feasible space that have not yet been visited and that may potentially contain better values of the objective function (exploration of the feasible space). This paper proposes a new global optimi…
AI enhanced data assimilation and uncertainty quantification applied to Geological Carbon Storage
Faculty of Civil Engineering；Geosciences, TU Delft, Stevinweg 1, 2628 CN Delft, Netherlands；Centrum Wiskunde \& Informatica, Science Park 123, 1098 XG Amsterdam, Netherlands；Mathematical Institute, Utrecht University, Utrecht, Netherlands；Department of Earth Science；Engineering, Imperial College London, London, United Kingdom；Department of Energy Resources Engineering, Stanford University, CA, USA
University of New South Wales（新南威尔士大学）；The University of Sydney（悉尼大学）；Indian Institute of Technology Guwahati（印度理工学院古瓦哈提分校）；SRM Institute of Science and Technology（SRM科学技术学院）；Indian Institute of Technology Delhi（印度理工学院德里分校）
Due to the need for robust uncertainty quantification, Bayesian neural learning has gained attention in the era of deep learning and big data. Markov Chain Monte-Carlo (MCMC) methods typically implement Bayesian inference which faces several challenges given a large number of parameters, complex and multimodal posterior distributions, and computational complexity of large neural network models. Parallel tempering MCMC addresses some of these limitations given that they can sample multimodal posterior distributions and utilize high-performance computing. However, certain challenges remain given…
Data assimilation is widely used to improve flood forecasting capability, especially through parameter inference requiring statistical information on the uncertain input parameters (upstream discharge, friction coefficient) as well as on the variability of the water level and its sensitivity with respect to the inputs. For particle filter or ensemble Kalman filter, stochastically estimating probability density function and covariance matrices from a Monte Carlo random sampling requires a large ensemble of model evaluations, limiting their use in real-time application. To tackle this issue, fas…
Graz University of Technology（格拉茨工业大学）；Graz Center for Computational Engineering（格拉茨计算工程中心）；University Medicine Essen（埃森大学医学院）；Norwegian University of Science and Technology (NTNU)（挪威科技大学）
Inhomogeneities in the aortic wall can lead to localized stress accumulations, possibly initiating dissection. In many cases, a dissection results from pathological changes such as fragmentation or loss of elastic fibers. But it has been shown that even the healthy aortic wall has an inherent heterogeneous microstructure. Some parts of the aorta are particularly susceptible to the development of inhomogeneities due to pathological changes, however, the distribution in the aortic wall and the spatial extent, such as size, shape, and type, are difficult to predict. Motivated by this observation,…
