贝叶斯设计的Laplace近似Nesterov辅助随机梯度法Optimization
Nesterov-aided Stochastic Gradient Methods using Laplace Approximation for Bayesian Design Optimization
摘要:
最优试验设计(OED)的首要任务是寻找最佳试验装置。在这里,我们专注于贝叶斯试验设计问题,找到设置,最大限度地提高香农预期的信息增益。我们使用随机梯度下降和它的加速对应,采用Nesterov的方法,解决最优化问题在牛津英语词典。我们采用重新启动技术,最初提出的加速确定性优化,以改善随机最优化方法。我们结合这些最优化方法与三个估计的目标函数:双环蒙特卡洛估计(DLMC),蒙特卡洛估计使用拉普拉斯近似的后验分布(MCLA)和双环蒙特卡罗估计与基于拉普拉斯的重要性抽样(DLMCIS)。同时使用随机梯度法和基于拉普拉斯的估计器允许我们使用昂贵且复杂的模型,例如需要求解偏微分方程(PDE)的模型。从理论的角度,我们推导出一个显式公式来计算蒙特卡罗方法,包括MCLA和DLMCIS的梯度估计。从计算的角度来看,我们研究了四个例子:三个基于解析函数和一个使用有限元法。最后一个例子是基于完整电极模型的电阻抗断层成像实验。在这些示例中,使用MCLA的加速随机梯度下降方法收敛到局部最大值,模型评估比使用DLMC的梯度下降少五个数量级。
英文摘要:
Finding the best setup for experiments is the primary concern for Optimal Experimental Design (OED). Here, we focus on the Bayesian experimental design problem of finding the setup that maximizes the Shannon expected information gain. We use the stochastic gradient descent and its accelerated counterpart, which employs Nesterov's method, to solve the optimization problem in OED. We adapt a restart technique, originally proposed for the acceleration in deterministic optimization, to improve stochastic optimization methods. We combine these optimization methods with three estimators of the objective function: the double-loop Monte Carlo estimator (DLMC), the Monte Carlo estimator using the Laplace approximation for the posterior distribution (MCLA) and the double-loop Monte Carlo estimator with Laplace-based importance sampling (DLMCIS). Using stochastic gradient methods and Laplace-based estimators together allows us to use expensive and complex models, such as those that require solving partial differential equations (PDEs). From a theoretical viewpoint, we derive an explicit formula to compute the gradient estimator of the Monte Carlo methods, including MCLA and DLMCIS. From a computational standpoint, we study four examples: three based on analytical functions and one using the finite element method. The last example is an electrical impedance tomography experiment based on the complete electrode model. In these examples, the accelerated stochastic gradient descent method using MCLA converges to local maxima with up to five orders of magnitude fewer model evaluations than gradient descent with DLMC.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。