We introduce a method to construct a stochastic surrogate model from the results of dimensionality reduction in forward uncertainty quantification. The hypothesis is that the high-dimensional input augmented by the output of a computational model admits a low-dimensional representation. This assumption can be met by numerous uncertainty quantification applications with physics-based computational models. The proposed approach differs from a sequential application of dimensionality reduction followed by surrogate modeling, as we "extract" a surrogate model from the results of dimensionality reduction in the input-output space. This feature becomes desirable when the input space is genuinely high-dimensional. The proposed method also diverges from the Probabilistic Learning on Manifold, as a reconstruction mapping from the feature space to the input-output space is circumvented. The final product of the proposed method is a stochastic simulator that propagates a deterministic input into a stochastic output, preserving the convenience of a sequential "dimensionality reduction + Gaussian process regression" approach while overcoming some of its limitations. The proposed method is demonstrated through two uncertainty quantification problems characterized by high-dimensional input uncertainties.
We are interested in the development of surrogate models for uncertainty quantification and propagation in problems governed by stochastic PDEs using a deep convolutional encoder-decoder network in a similar fashion to approaches considered in deep learning for image-to-image regression tasks. Since normal neural networks are data intensive and cannot provide predictive uncertainty, we propose a Bayesian approach to convolutional neural nets. A recently introduced variational gradient descent algorithm based on Stein's method is scaled to deep convolutional networks to perform approximate Baye…
National Cheng Kung University（国立成功大学）；Wells Fargo（富国银行）；Georgia Institute of Technology（佐治亚理工学院）
Global optimization of expensive functions has important applications in physical and computer experiments. It is a challenging problem to develop efficient optimization scheme, because each function evaluation can be costly and the derivative information of the function is often not available. We propose a novel global optimization framework using adaptive Radial Basis Functions (RBF) based surrogate model via uncertainty quantification. The framework consists of two iteration steps. It first employs an RBF-based Bayesian surrogate model to approximate the true function, where the parameters …
Polynomial chaos and Gaussian process emulation are methods for surrogate-based uncertainty quantification, and have been developed independently in their respective communities over the last 25 years. Despite tackling similar problems in the field, to our knowledge there has yet to be a critical comparison of the two approaches in the literature. We begin by providing a detailed description of polynomial chaos and Gaussian process approaches for building a surrogate model of a black-box function. The accuracy of each surrogate method is then tested and compared for two simulators used in indu…
University of Amsterdam（阿姆斯特丹大学）；ITMO University（圣光机大学）
In-Stent Restenosis is a recurrence of coronary artery narrowing due to vascular injury caused by balloon dilation and stent placement. It may lead to the relapse of angina symptoms or to an acute coronary syndrome. An uncertainty quantification of a model for In-Stent Restenosis with four uncertain parameters (endothelium regeneration time, the threshold strain for smooth muscle cells bond breaking, blood flow velocity and the percentage of fenestration in the internal elastic lamina) is presented. Two quantities of interest were studied, namely the average cross-sectional area and the maximu…
Department of Biostatistics, University of Oslo（奥斯陆大学生物统计系）；Helsinki Institute for Information Technology HIIT, Department of Computer Science, Aalto University（赫尔辛基信息技术研究所，阿尔托大学计算机科学系）
The computational efficiency of approximate Bayesian computation (ABC) has been improved by using surrogate models such as Gaussian processes (GP). In one such promising framework the discrepancy between the simulated and observed data is modelled with a GP which is further used to form a model-based estimator for the intractable posterior. In this article we improve this approach in several ways. We develop batch-sequential Bayesian experimental design strategies to parallellise the expensive simulations. In earlier work only sequential strategies have been used. Current surrogate-based ABC m…
Data assimilation is widely used to improve flood forecasting capability, especially through parameter inference requiring statistical information on the uncertain input parameters (upstream discharge, friction coefficient) as well as on the variability of the water level and its sensitivity with respect to the inputs. For particle filter or ensemble Kalman filter, stochastically estimating probability density function and covariance matrices from a Monte Carlo random sampling requires a large ensemble of model evaluations, limiting their use in real-time application. To tackle this issue, fas…
Seismic fragility curves have been introduced as key components of Seismic Probabilistic Risk Assessment studies. They express the probability of failure of mechanical structures conditional to a seismic intensity measure and must take into account the inherent uncertainties in such studies, the so-called epistemic uncertainties (i.e. coming from the uncertainty on the mechanical parameters of the structure) and the aleatory uncertainties (i.e. coming from the randomness of the seismic ground motions). For simulation-based approaches we propose a methodology to build and calibrate a Gaussian p…
Uncertainty Quantification for Machine Learning-Based Prediction: A Polynomial Chaos Expansion Approach for Joint Model and Input Uncertainty Propagation
School of Mechanical Engineering（机械工程学院）；Purdue University（普渡大学）
A Bayesian latent Gaussian process framework for aerodynamic uncertainty quantification
Aerospace Engineering, The Pennsylvania State University, University Park, PA 16802（航空工程，宾夕法尼亚州立大学，University Park分校，PA 16802）；Aerospace Engineering and the Institute of Computational and Data Science (ICDS), The Pennsylvania State University, University Park, PA 16802（航空工程和计算与数据科学研究所（ICDS），宾夕法尼亚州立大学，University Park分校，PA 16802）
Coventry University（考文垂大学）；Malayer University（马拉耶尔大学）；Cranfield University（克兰菲尔德大学）；The University of Warwick（华威大学）
A computational method based on the non-linear Gaussian process (GP), known as deep Gaussian processes (deep GPs) for uncertainty quantification & propagation in modelling of flow through heterogeneous porous media is presented. The method is also used for reducing dimensionality of model output and consequently emulating highly complex relationship between hydrogeological properties and reduced order fluid velocity field in a tractable manner. Deep GPs are multi-layer hierarchical generalisations of GPs with multiple, infinitely wide hidden layers that are very efficient models for deep learn…
We introduce a methodology for nonlinear inverse problems using a variational Bayesian approach where the unknown quantity is a spatial field. A structured Bayesian Gaussian process latent variable model is used both to construct a low-dimensional generative model of the sample-based stochastic prior as well as a surrogate for the forward evaluation. Its Bayesian formulation captures epistemic uncertainty introduced by the limited number of input and output examples, automatically selects an appropriate dimensionality for the learned latent representation of the data, and rigorously propagates…
University of New South Wales（新南威尔士大学）；The University of Sydney（悉尼大学）；Indian Institute of Technology Guwahati（印度理工学院古瓦哈提分校）；SRM Institute of Science and Technology（SRM科学技术学院）；Indian Institute of Technology Delhi（印度理工学院德里分校）
Due to the need for robust uncertainty quantification, Bayesian neural learning has gained attention in the era of deep learning and big data. Markov Chain Monte-Carlo (MCMC) methods typically implement Bayesian inference which faces several challenges given a large number of parameters, complex and multimodal posterior distributions, and computational complexity of large neural network models. Parallel tempering MCMC addresses some of these limitations given that they can sample multimodal posterior distributions and utilize high-performance computing. However, certain challenges remain given…
Graz University of Technology（格拉茨工业大学）；Graz Center for Computational Engineering（格拉茨计算工程中心）；University Medicine Essen（埃森大学医学院）；Norwegian University of Science and Technology (NTNU)（挪威科技大学）
Inhomogeneities in the aortic wall can lead to localized stress accumulations, possibly initiating dissection. In many cases, a dissection results from pathological changes such as fragmentation or loss of elastic fibers. But it has been shown that even the healthy aortic wall has an inherent heterogeneous microstructure. Some parts of the aorta are particularly susceptible to the development of inhomogeneities due to pathological changes, however, the distribution in the aortic wall and the spatial extent, such as size, shape, and type, are difficult to predict. Motivated by this observation,…
University of New South Wales（新南威尔士大学）；University of Sydney（悉尼大学）；SRM Institute of Science and Technology（SRM科学技术研究所）
The complex and computationally expensive nature of landscape evolution models pose significant challenges in the inference and optimisation of unknown parameters. Bayesian inference provides a methodology for estimation and uncertainty quantification of unknown model parameters. In our previous work, we developed parallel tempering Bayeslands as a framework for parameter estimation and uncertainty quantification for the Badlands landscape evolution model. Parallel tempering Bayeslands features high-performance computing with dozens of processing cores running in parallel to enhance computatio…
SCORE: A 1D Reparameterization Technique to Break Bayesian Optimization's Curse of Dimensionality
Laboratoire de Physique des Interfaces et des Couches Minces (LPICM)（界面与薄膜物理实验室（LPICM））；Ecole Polytechnique, Institut Polytechnique de Paris (IP Paris)（巴黎综合理工学院，巴黎理工学院（IP Paris））；Institut Photovoltaïque d’Île-de-France (IPVF)（法兰西岛光伏研究所（IPVF））
In statistical applications, it is common to encounter parameters supported on a varying or unknown dimensional space. Examples include the fused lasso regression, the matrix recovery under an unknown low rank, etc. Despite the ease of obtaining a point estimate via the optimization, it is much more challenging to quantify their uncertainty -- in the Bayesian framework, a major difficulty is that if assigning the prior associated with a $p$-dimensional measure, then there is zero posterior probability on any lower-dimensional subset with dimension $d<p$; to avoid this caveat, one needs to choo…
Uncertainty quantification in mechanics: A unified Bayesian perspective
Courant Institute of Mathematical Sciences, New York University（纽约大学柯朗数学科学研究所）；Division of Applied Mathematics, Brown University（布朗大学应用数学系）；Institute of Applied Mechanics, Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡大学应用力学研究所）；Institute of Biomechanics, Graz University of Technology（格拉茨工业大学生物力学研究所）；Department of Structural Engineering, Norwegian University of Science and Technology (NTNU)（挪威科技大学结构工程系）；Department of Mechanical Engineering, Stanford University（斯坦福大学机械工程系）
Pohang University of Science and Technology (POSTECH)（浦项科技大学）
We develop a novel deep learning method for uncertainty quantification in stochastic partial differential equations based on Bayesian neural network (BNN) and Hamiltonian Monte Carlo (HMC). A BNN efficiently learns the posterior distribution of the parameters in deep neural networks by performing Bayesian inference on the network parameters. The posterior distribution is efficiently sampled using HMC to quantify uncertainties in the system. Several numerical examples are shown for both forward and inverse problems in high dimension to demonstrate the effectiveness of the proposed method for un…
This work develops rigorous theoretical basis for the fact that deep Bayesian neural network (BNN) is an effective tool for high-dimensional variable selection with rigorous uncertainty quantification. We develop new Bayesian non-parametric theorems to show that a properly configured deep BNN (1) learns the variable importance effectively in high dimensions, and its learning rate can sometimes "break" the curse of dimensionality. (2) BNN's uncertainty quantification for variable importance is rigorous, in the sense that its 95% credible intervals for variable importance indeed covers the truth…
We investigate a machine learning approach to option Greeks approximation based on Gaussian process (GP) surrogates. The method takes in noisily observed option prices, fits a nonparametric input-output map and then analytically differentiates the latter to obtain the various price sensitivities. Our motivation is to compute Greeks in cases where direct computation is expensive, such as in local volatility models, or can only ever be done approximately. We provide a detailed analysis of numerous aspects of GP surrogates, including choice of kernel family, simulation design, choice of trend fun…
The quantification of uncertainties of computer simulations due to input parameter uncertainties is paramount to assess a model's credibility. For computationally expensive simulations, this is often feasible only via surrogate models that are learned from a small set of simulation samples. The surrogate models are commonly chosen and deemed trustworthy based on heuristic measures, and substituted for the simulation in order to approximately propagate the simulation input uncertainties to the simulation output. In the process, the contribution of the uncertainties of the surrogate itself to th…
Bayesian inverse modeling is important for a better understanding of hydrological processes. However, this approach can be computationally demanding, as it usually requires a large number of model evaluations. To address this issue, one can take advantage of surrogate modeling techniques. Nevertheless, when approximation error of the surrogate model is neglected, the inversion result will be biased. In this paper, we develop a surrogate-based Bayesian inversion framework that explicitly quantifies and gradually reduces the approximation error of the surrogate. Specifically, two strategies are …
Bangladesh University of Engineering and Technology（孟加拉国工程技术大学）；Jashore University of Science and Technology（杰索尔科技大学）；Indian Institute of Technology Delhi（印度理工学院德里分校）；University of Bristol（布里斯托大学）；Missouri University of Science and Technology（密苏里科技大学）
Industry 4.0 targets the conversion of the traditional industries into intelligent ones through technological revolution. This revolution is only possible through innovation, optimization, interconnection, and rapid decision-making capability. Numerical models are believed to be the key components of Industry 4.0, facilitating quick decision-making through simulations instead of costly experiments. However, numerical investigation of precise, high-fidelity models for optimization or decision-making is usually time-consuming and computationally expensive. In such instances, data-driven surrogat…
Surrogate Modelling of Proton Dose with Monte Carlo Dropout Uncertainty Quantification
Institute for Mathematical Innovation University of Bath, Bath, UK（数学创新研究所 英国巴斯大学）；Department of Mathematical Sciences University of Bath, Bath, UK（数学科学系 英国巴斯大学）
Computational models of the human head are promising tools for estimating the impact-induced response of brain, and thus play an important role in the prediction of traumatic brain injury. Modern biofidelic head model simulations are associated with very high computational cost, and high-dimensional inputs and outputs, which limits the applicability of traditional uncertainty quantification (UQ) methods on these systems. In this study, a two-stage, data-driven manifold learning-based framework is proposed for UQ of computational head models. This framework is demonstrated on a 2D subject-speci…
