In this paper, we focus on developing efficient sensitivity analysis methods for a computationally expensive objective function $f(x)$ in the case that the minimization of it has just been performed. Here "computationally expensive" means that each of its evaluation takes significant amount of time, and therefore our main goal to use a small number of function evaluations of $f(x)$ to further infer the sensitivity information of these different parameters. Correspondingly, we consider the optimization procedure as an adaptive experimental design and re-use its available function evaluations as the initial design points to establish a surrogate model $s(x)$ (or called response surface). The sensitivity analysis is performed on $s(x)$, which is an lieu of $f(x)$. Furthermore, we propose a new local multivariate sensitivity measure, for example, around the optimal solution, for high dimensional problems. Then a corresponding "objective-oriented experimental design" is proposed in order to make the generated surrogate $s(x)$ better suitable for the accurate calculation of the proposed specific local sensitivity quantities. In addition, we demonstrate the better performance of the Gaussian radial basis function interpolator over Kriging in our cases, which are of relatively high dimensionality and few experimental design points. Numerical experiments demonstrate that the optimization procedure and the "objective-oriented experimental design" behavior much better than the classical Latin Hypercube Design. In addition, the performance of Kriging is not as good as Gaussian RBF, especially in the case of high dimensional problems.
Variational Inference with NoFAS: Normalizing Flow with Adaptive Surrogate for Computationally Expensive Models
University of Notre Dame(圣母大学)
从数据中快速推断数值模型参数是生成广泛应用的预测模型的重要前提。使用基于采样的方法,如马尔可夫链蒙特卡罗可能会变得棘手时,每个可能性评估是计算昂贵的。变分推理与归一化流相结合的新方法的特征在于计算成本仅随潜在变量空间的维度线性增长,并且依赖于基于梯度的优化而不是采样,为关于模型参数的贝叶斯推理提供了更有效的方法。此外,通过用离线训练的代理模型(诸如神经网络)替换真实模型,可以减轻频繁评估昂贵的可能性的成本。然而,这种方法可能会产生显着的偏见时,替代品是不够准确的后验模式。为了在不牺牲推理精度的前提下降低计算成本,本文提出了一种基于自适应代理的流标准化算法(Normalizing Flow with Adaptive Surrogate,NoFAS),该算法通过交替更新流标准化参数和代理模型参数来优化流模型。我们还提出了一个有效的样本加权方案的代理模型训练,保持全球的准确性,同时有效地捕捉高后验密度区域。我们展示了NoFAS对各种基准的推理和计算优势,包括基础模型缺乏可识别性的情况。用于本研究的源代码和数值实验可在https://github.com/cedricwangyu/NoFAS上获得。
Uncertainty-Aware Surrogate-based Amortized Bayesian Inference for Computationally Expensive Models
Department of Stochastic Simulation and Safety Research for Hydrosystems(随机模拟与安全研究部门);Cluster of Excellence SimTech(卓越中心SimTech);University of Stuttgart(斯图加特大学);Department of Statistics(统计学系);TU Dortmund University(多特蒙德技术大学)
Surrogate-assisted Bayesian inversion for landscape and basin evolution models
University of New South Wales(新南威尔士大学);University of Sydney(悉尼大学);SRM Institute of Science and Technology(SRM科学技术研究所)
景观演化模型的复杂性和计算成本高的性质在未知参数的推断和优化方面提出了重大挑战。贝叶斯推断提供了一种用于未知模型参数的估计和不确定性量化的方法。在我们以前的工作中,我们开发了并行回火贝叶斯作为参数估计和不确定性量化的荒地景观演化模型的框架。并行回火Bayeslands具有高性能计算功能,具有数十个并行运行的处理核心,以提高计算效率。虽然我们使用并行计算,但该过程仍然具有计算挑战性,因为需要绘制和评估数千个样本。\textcolor{black}{In large-scale landscape and basin evolution problems, a single model evaluation can take from several minutes to hours, and in some instances, even days. Surrogate-assisted optimisation has been used for several computationally expensive engineering problems which motivate its use in optimisation and inference of complex geoscientific models.}代理模型的使用可以通过开发计算成本低廉的模型来模拟昂贵的模…
Multifidelity-Augmented Gaussian Process Inputs for Surrogate Modeling from Scarce Data
School of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA(航空航天工程学院,佐治亚理工学院,亚特兰大,GA);School of Computational Science & Engineering, Georgia Institute of Technology, Atlanta, GA(计算科学与工程学院,佐治亚理工学院,亚特兰大,GA);Air Force Research Lab, Wright-Patterson AFB, Dayton, Ohio(空军研究实验室,威格伍德-帕特森空军基地,代顿,俄亥俄)
Improving Pareto Set Learning for Expensive Multi-objective Optimization via Stein Variational Hypernetworks
College of Engineering and Computer Science, VinUniversity(VinUniversity工程与计算机科学学院);The Ohio State University(俄亥俄州立大学);Singapore University of Technology and Design(新加坡科技设计大学)