arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.19670stat.COstat.ME
收藏

SULA:通过漂移-扩散饱和实现的鲁棒朗之万采样

SULA: Robust Langevin Sampling via Drift-Diffusion Saturation

Indian Institute of Technology Kanpur(坎普尔印度理工学院)

Ritik Soni, Dootika Vats

AI总结:

针对ULA和SGLD在非利普希茨漂移目标分布下的脆弱性问题,提出SULA算法,结合饱和梯度漂移与状态依赖扩散,保证鲁棒性与收敛性,还开发其随机梯度变体SG-SULA,实验验证了二者的优势。

摘要:

未调整朗之万算法(ULA)及其随机梯度变体SGLD是现代贝叶斯机器学习的核心算法,但当目标分布具有非利普希茨漂移时,它们可能变得脆弱。在这种情况下,期望平方跳距会快速增长,导致数值不稳定或需要过小的步长。我们提出饱和未调整朗之万算法(SULA),该算法将饱和梯度漂移与梯度感知的状态依赖扩散相结合,同时保持与朗之万扩散的一阶一致性。漂移和扩散项的选择保证了单步平方更新与梯度无关,因此SULA对步长的选择具有鲁棒性,并保持估计和预测质量。我们提供了充分条件,在这些条件下,步长为$h$的SULA以标准$O(h)$速率弱收敛到朗之万扩散,SULA链是几何遍历的,且其不变测度相对于目标的偏差为$O(h)$。此外,我们开发了SULA的随机梯度版本SG-SULA,该版本在梯度噪声为高斯分布时,使用高斯CDF校正对随机梯度进行重新缩放。对各种示例的数值实验表明,SULA对调参具有鲁棒性,且SG-SULA的预测性能有所提升。

英文摘要:

The unadjusted Langevin algorithm (ULA) and its stochastic-gradient variant, SGLD, are workhorses of modern Bayesian machine learning, but can be fragile when faced with a target distribution possessing a non-Lipschitz drift. In such settings, the expected squared jump distance can grow rapidly, leading to numerical instability or the need for prohibitively small step-sizes. We propose the saturated unadjusted Langevin algorithm (SULA) that pairs a saturated gradient drift with a gradient-informed state-dependent diffusion, while preserving first-order consistency with the Langevin diffusion. The drift and diffusion terms are chosen to guarantee that the one-step squared update is independent of the gradient. Consequently, SULA is robust to the choice of step-size and preserves estimation and prediction quality. We provide sufficient conditions under which SULA with a step-size $h$ converges weakly to the Langevin diffusion at the standard $O(h)$ rate, the SULA chain is geometrically ergodic, and that the bias of its invariant measure relative to the target is $O(h)$. Further, we develop SG-SULA, a stochastic-gradient version of SULA that rescales the stochastic gradient using a Gaussian CDF correction under Gaussianity of the gradient noise. Numerical experiments on various examples demonstrate SULA's robustness to tuning and SG-SULA's improved predictive performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑