AMAGOLD：有效随机梯度的摊余Metropolis平差MCMC
AMAGOLD: Amortized Metropolis Adjustment for Efficient Stochastic Gradient MCMC
摘要：
随机梯度哈密顿蒙特卡罗（SGHMC）是一种有效的连续分布采样方法。它是HMC的一个更快的替代方案：SGHMC在每次迭代时只使用子样本，而不是使用整个数据集。这提高了性能，但引入了偏差，可能导致SGHMC收敛到错误的分布。人们可以使用衰减到零的步长来防止这种情况，但是这样的步长调度可以大大减慢收敛。为了解决这种紧张局势，我们提出了一种新的二阶SG-MCMC算法-AMAGOLD-很少使用Metropolis-Hastings（M-H）校正来消除偏差。不频繁的更正摊销了它们的成本。我们证明AMAGOLD收敛到目标分布与一个固定的，而不是一个递减的，步长，它的收敛速度是最多一个常数因子慢于一个完整的批量基线。我们经验证明AMAGOLD的有效性合成分布，贝叶斯逻辑回归，贝叶斯神经网络。
英文摘要：
Stochastic gradient Hamiltonian Monte Carlo (SGHMC) is an efficient method for sampling from continuous distributions. It is a faster alternative to HMC: instead of using the whole dataset at each iteration, SGHMC uses only a subsample. This improves performance, but introduces bias that can cause SGHMC to converge to the wrong distribution. One can prevent this using a step size that decays to zero, but such a step size schedule can drastically slow down convergence. To address this tension, we propose a novel second-order SG-MCMC algorithm---AMAGOLD---that infrequently uses Metropolis-Hastings (M-H) corrections to remove bias. The infrequency of corrections amortizes their cost. We prove AMAGOLD converges to the target distribution with a fixed, rather than a diminishing, step size, and that its convergence rate is at most a constant factor slower than a full-batch baseline. We empirically demonstrate AMAGOLD's effectiveness on synthetic distributions, Bayesian logistic regression, and Bayesian neural networks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
