Estimating the probability of failure for complex real-world systems using high-fidelity computational models is often prohibitively expensive, especially when the probability is small. Exploiting low-fidelity models can make this process more feasible, but merging information from multiple low-fidelity and high-fidelity models poses several challenges. This paper presents a robust multi-fidelity surrogate modeling strategy in which the multi-fidelity surrogate is assembled using an active learning strategy using an on-the-fly model adequacy assessment set within a subset simulation framework for efficient reliability analysis. The multi-fidelity surrogate is assembled by first applying a Gaussian process correction to each low-fidelity model and assigning a model probability based on the model's local predictive accuracy and cost. Three strategies are proposed to fuse these individual surrogates into an overall surrogate model based on model averaging and deterministic/stochastic model selection. The strategies also dictate which model evaluations are necessary. No assumptions are made about the relationships between low-fidelity models, while the high-fidelity model is assumed to be the most accurate and most computationally expensive model. Through two analytical and two numerical case studies, including a case study evaluating the failure probability of Tristructural isotropic-coated (TRISO) nuclear fuels, the algorithm is shown to be highly accurate while drastically reducing the number of high-fidelity model calls (and hence computational cost).
Multi-fidelity Gaussian process surrogate modeling for regression problems in physics
School of Computation, Information and Technology, Technische Universität München（慕尼黑工业大学计算、信息与技术学院）；Institute of System Dynamics and Control, German Aerospace Center (DLR)（德国航空航天中心系统动力学与控制研究所）
Accurately modelling the Earth's climate has widespread applications ranging from forecasting local weather to understanding global climate change. Low-fidelity simulations of climate phenomena are readily available, but high-fidelity simulations are expensive to obtain. We therefore investigate the potential of Gaussian process-based multi-fidelity surrogate modelling as a way to produce high-fidelity climate predictions at low cost. Specifically, our model combines the predictions of a low-fidelity Global Climate Model (GCM) and those of a high-fidelity Regional Climate Model (RCM) to produc…
Multifidelity-Augmented Gaussian Process Inputs for Surrogate Modeling from Scarce Data
School of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA（航空航天工程学院，佐治亚理工学院，亚特兰大，GA）；School of Computational Science & Engineering, Georgia Institute of Technology, Atlanta, GA（计算科学与工程学院，佐治亚理工学院，亚特兰大，GA）；Air Force Research Lab, Wright-Patterson AFB, Dayton, Ohio（空军研究实验室，威格伍德-帕特森空军基地，代顿，俄亥俄）
A Unified Hierarchical Multi-Task Multi-Fidelity Framework for Data-Efficient Surrogate Modeling in Manufacturing
Department of Mechanical Science and Engineering, University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校机械科学与工程系）；Department of Mechanical Engineering, University of Michigan（密歇根大学机械工程系）
On the performance of multi-fidelity and reduced-dimensional neural emulators for inference of physiological boundary conditions
Stanford University（斯坦福大学）；Centro di Ricerca Matematica Ennio De Giorgi, Scuola Normale Superiore（比萨高等师范学院恩尼奥·德·乔治数学研究中心）；University of Notre Dame（圣母大学）
University of Victoria（维多利亚大学）；Department of Civil Engineering（土木工程系）；Energy and Cities Group（能源与城市小组）
Fast machine learning-based surrogate models are trained to emulate slow, high-fidelity engineering simulation models to accelerate engineering design tasks. This introduces uncertainty as the surrogate is only an approximation of the original model. Bayesian methods can quantify that uncertainty, and deep learning models exist that follow the Bayesian paradigm. These models, namely Bayesian neural networks and Gaussian process models, enable us to give predictions together with an estimate of the model's uncertainty. As a result we can derive uncertainty-aware surrogate models that can automa…
The Gaussian Process (GP)-based surrogate model has the inherent capability of capturing the anomaly arising from limited data, lack of data, missing data, and data inconsistencies (noisy/erroneous data) present in the modeling and simulation component of the digital twin framework, specifically for the accident tolerant fuel (ATF) concepts. However, GP will not be very accurate when we have limited high-fidelity (experimental) data. In addition, it is challenging to apply higher dimensional functions (>20-dimensional function) to approximate predictions with the GP. Furthermore, noisy data or…
In computational fluid dynamics, there is an inevitable trade off between accuracy and computational cost. In this work, a novel multi-fidelity deep generative model is introduced for the surrogate modeling of high-fidelity turbulent flow fields given the solution of a computationally inexpensive but inaccurate low-fidelity solver. The resulting surrogate is able to generate physically accurate turbulent realizations at a computational cost magnitudes lower than that of a high-fidelity simulation. The deep generative model developed is a conditional invertible neural network, built with normal…
Bifidelity Karhunen-Loève Expansion Surrogate with Active Learning for Random Fields
Department of Mechanical Engineering University of Michigan（机械工程系密歇根大学）；Sandia National Laboratories（沙伊国家实验室）；Guggenheim School of Aerospace Engineering Georgia Institute of Technology（盖茨基金会航空航天工程学院佐治亚理工学院）
University of British Columbia（不列颠哥伦比亚大学）；Inverted AI Ltd.（Inverted AI有限公司）；University of Oxford（牛津大学）；Convergent Manufacturing Technologies Inc.（Convergent制造技术公司）；Mila（米拉）
We present a framework for automatically structuring and training fast, approximate, deep neural surrogates of stochastic simulators. Unlike traditional approaches to surrogate modeling, our surrogates retain the interpretable structure and control flow of the reference simulator. Our surrogates target stochastic simulators where the number of random variables itself can be stochastic and potentially unbounded. Our framework further enables an automatic replacement of the reference simulator with the surrogate when undertaking amortized inference. The fidelity and speed of our surrogates allow…
University of California, San Diego（加州大学圣地亚哥分校）；Northeastern University（东北大学）
Science and engineering fields use computer simulation extensively. These simulations are often run at multiple levels of sophistication to balance accuracy and efficiency. Multi-fidelity surrogate modeling reduces the computational cost by fusing different simulation outputs. Cheap data generated from low-fidelity simulators can be combined with limited high-quality data generated by an expensive high-fidelity simulator. Existing methods based on Gaussian processes rely on strong assumptions of the kernel functions and can hardly scale to high-dimensional settings. We propose Multi-fidelity H…
We study the problem of learning generative adversarial networks (GANs) for a rare class of an unlabeled dataset subject to a labeling budget. This problem is motivated from practical applications in domains including security (e.g., synthesizing packets for DNS amplification attacks), systems and networking (e.g., synthesizing workloads that trigger high resource usage), and machine learning (e.g., generating images from a rare class). Existing approaches are unsuitable, either requiring fully-labeled datasets or sacrificing the fidelity of the rare class for that of the common classes. We pr…
Multi-fidelity methods leverage low-cost surrogate models to speed up computations and make occasional recourse to expensive high-fidelity models to establish accuracy guarantees. Because surrogate and high-fidelity models are used together, poor predictions by surrogate models can be compensated with frequent recourse to high-fidelity models. Thus, there is a trade-off between investing computational resources to improve the accuracy of surrogate models versus simply making more frequent recourse to expensive high-fidelity models; however, this trade-off is ignored by traditional modeling met…
Computational simulations with different fidelity have been widely used in engineering design. A high-fidelity (HF) model is generally more accurate but also more time-consuming than an low-fidelity (LF) model. To take advantages of both HF and LF models, multi-fidelity surrogate models that aim to integrate information from both HF and LF models have gained increasing popularity. In this paper, a multi-fidelity surrogate model based on support vector regression named as Co_SVR is developed by combining HF and LF models. In Co_SVR, a kernel function is used to map the map the difference betwee…
We present a probabilistic deep learning methodology that enables the construction of predictive data-driven surrogates for stochastic systems. Leveraging recent advances in variational inference with implicit distributions, we put forth a statistical inference framework that enables the end-to-end training of surrogate models on paired input-output observations that may be stochastic in nature, originate from different information sources of variable fidelity, or be corrupted by complex noise processes. The resulting surrogates can accommodate high-dimensional inputs and outputs and are able …
University of Bristol（布里斯托大学）；AXA（安盛）；Sorbonne Université（索邦大学）；CNRS（法国国家科学研究中心）；Polish Academy of Science（波兰科学院）
This paper analyses the fundamental ingredients behind surrogate explanations to provide a better understanding of their inner workings. We start our exposition by considering global surrogates, describing the trade-off between complexity of the surrogate and fidelity to the black-box being modelled. We show that transitioning from global to local - reducing coverage - allows for more favourable conditions on the Pareto frontier of fidelity-complexity of a surrogate. We discuss the interplay between complexity, fidelity and coverage, and consider how different user needs can lead to problem fo…
Peking University（北京大学）；Peng Cheng Laboratory（鹏城实验室）；Southern University of Science and Technology（南方科技大学）；China University of Geosciences（中国地质大学）
Particle settling in inclined channels is an important phenomenon that occurs during hydraulic fracturing of shale gas production. Generally, in order to accurately simulate the large-scale (field-scale) proppant transport process, constructing a fast and accurate sub-scale proppant settling model, or surrogate model, becomes a critical issue, since mapping between physical parameters and proppant settling velocity is complex. Previously, particle settling has usually been investigated via high-fidelity experiments and meso-scale numerical simulations, both of which are time-consuming. In this…
Defense Innovation Institute, Chinese Academy of Military Science（中国人民解放军军事科学院国防科技创新研究院）；College of Aerospace Science and Engineering, National University of Defense Technology（国防科技大学空天科学学院）
Temperature field prediction is of great importance in the thermal design of systems engineering, and building the surrogate model is an effective way for the task. Generally, large amounts of labeled data are required to guarantee a good prediction performance of the surrogate model, especially the deep learning model, which have more parameters and better representational ability. However, labeled data, especially high-fidelity labeled data, are usually expensive to obtain and sometimes even impossible. To solve this problem, this paper proposes a pithy deep multi-fidelity model (DMFM) for t…
Balancing Accuracy and Speed: A Multi-Fidelity Ensemble Kalman Filter with a Machine Learning Surrogate Model
Delft Institute of Applied Mathematics, Delft University of Technology, Delft, The Netherlands（代尔夫特应用数学研究所，代尔夫特理工大学，代尔夫特，荷兰）；Deltares, Delft, The Netherlands（Deltares，代尔夫特，荷兰）
Politecnico di Milano（米兰理工大学）；University of Twente（特文特大学）；École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）
When evaluating quantities of interest that depend on the solutions to differential equations, we inevitably face the trade-off between accuracy and efficiency. Especially for parametrized, time dependent problems in engineering computations, it is often the case that acceptable computational budgets limit the availability of high-fidelity, accurate simulation data. Multi-fidelity surrogate modeling has emerged as an effective strategy to overcome this difficulty. Its key idea is to leverage many low-fidelity simulation data, less accurate but much faster to compute, to improve the approximati…
University of Notre Dame（圣母大学）；Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）
Simulations of high energy density physics are expensive, largely in part for the need to produce non-local thermodynamic equilibrium opacities. High-fidelity spectra may reveal new physics in the simulations not seen with low-fidelity spectra, but the cost of these simulations also scale with the level of fidelity of the opacities being used. Neural networks are capable of reproducing these spectra, but neural networks need data to to train them which limits the level of fidelity of the training data. This paper demonstrates that it is possible to reproduce high-fidelity spectra with median e…
Data assimilation presents computational challenges because many high-fidelity models must be simulated. Various deep-learning-based surrogate modeling techniques have been developed to reduce the simulation costs associated with these applications. However, to construct data-driven surrogate models, several thousand high-fidelity simulation runs may be required to provide training samples, and these computations can make training prohibitively expensive. To address this issue, in this work we present a framework where most of the training simulations are performed on coarsened geomodels. Thes…
Compute-optimal data scaling for neural surrogates via multi-fidelity training
Institute for Machine Learning, JKU Linz（林茨约翰内斯·开普勒大学机器学习研究所）；Mistral AI（Mistral AI公司）；Technical University of Munich（慕尼黑工业大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）
Parameter identification for marine ecosystem models is important for the assessment and validation of marine ecosystem models against observational data. The surrogate-based optimization (SBO) is a computationally efficient method to optimize complex models. SBO replaces the computationally expensive (high-fidelity) model by a surrogate constructed from a less accurate but computationally cheaper (low-fidelity) model in combination with an appropriate correction approach, which improves the accuracy of the low-fidelity model. To construct a computationally cheap low-fidelity model, we tested …
Causal Multi-fidelity Surrogate Forward and Inverse Models for ICF Implosions
Theoretical Division, Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室理论 division）；Computer, Computational, and Statistical Sciences Division, Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室计算机、计算与统计科学 division）
Meta-materials are an important emerging class of engineered materials in which complex macroscopic behaviour--whether electromagnetic, thermal, or mechanical--arises from modular substructure. Simulation and optimization of these materials are computationally challenging, as rich substructures necessitate high-fidelity finite element meshes to solve the governing PDEs. To address this, we leverage parametric modular structure to learn component-level surrogates, enabling cheaper high-fidelity simulation. We use a neural network to model the stored potential energy in a component given boundar…
