多峰分布模拟的轮廓随机梯度Langevin动力学算法
A Contour Stochastic Gradient Langevin Dynamics Algorithm for Simulations of Multi-modal Distributions
摘要：
我们提出了一种自适应加权随机梯度Langevin动态算法（SGLD），即所谓的轮廓随机梯度Langevin动态（CSGLD），用于大数据统计中的贝叶斯学习。该算法本质上是一个\emph{scalable dynamic importance sampler}，它自动\emph{flattens}的目标分布，这样的多模态分布的模拟可以大大方便。在理论上，我们证明了一个稳定性条件，并建立了自适应参数渐近收敛于{ \it唯一不动点}，而不考虑原能量函数的非凸性;我们还给出了加权平均估计的误差分析。经验上，CSGLD算法在包括CIFAR 10和CIFAR 100在内的多个基准数据集上进行了测试。数值实验结果表明，该算法在避免深度神经网络训练中的局部陷阱问题方面具有优越性。
英文摘要：
We propose an adaptively weighted stochastic gradient Langevin dynamics algorithm (SGLD), so-called contour stochastic gradient Langevin dynamics (CSGLD), for Bayesian learning in big data statistics. The proposed algorithm is essentially a \emph{scalable dynamic importance sampler}, which automatically \emph{flattens} the target distribution such that the simulation for a multi-modal distribution can be greatly facilitated. Theoretically, we prove a stability condition and establish the asymptotic convergence of the self-adapting parameter to a {\it unique fixed-point}, regardless of the non-convexity of the original energy function; we also present an error analysis for the weighted averaging estimators. Empirically, the CSGLD algorithm is tested on multiple benchmark datasets including CIFAR10 and CIFAR100. The numerical results indicate its superiority to avoid the local trap problem in training deep neural networks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
