arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.11929cs.LGmath.PR
收藏

总体极限之外带噪声SVGD的长时间渐近行为

Long-time asymptotics of noisy SVGD outside the population limit

Télécom Paris(巴黎电信学院);Microsoft Research(微软研究院)

Victor Priser, Pascal Bianchi, Adil Salim

AI总结:

该研究针对有限粒子情形下SVGD长时间渐近行为不明确的问题,分析带噪声SVGD变体的长期特性,证明其极限集良定义且随粒子数增加趋近目标分布,可避免方差崩塌,轨迹与McKean-Vlasov过程高度相似。

摘要:

Stein变分梯度下降(Stein Variational Gradient Descent,SVGD)是一种被广泛使用的采样算法,已成功应用于机器学习的多个领域。SVGD通过迭代移动一组相互作用的粒子(代表样本)来逼近目标分布。尽管近期已有针对SVGD及其变体复杂度的研究,但在粒子数量有限的情形下,它们的长时间渐近行为(即经过大量迭代后的行为)仍未被阐明。我们研究了一种带噪声的SVGD变体的长时间渐近行为。首先,我们证明了当粒子数较大时,带噪声SVGD的极限集是良定义的。随后我们对该极限集进行了刻画,表明其会随着粒子数增加而趋近于目标分布。尤其值得注意的是,带噪声SVGD可被证明能避免标准SVGD中出现的方差崩塌问题。我们的研究方法涉及证明带噪声SVGD的轨迹与McKean-Vlasov过程描述的轨迹高度相似。

英文摘要:

Stein Variational Gradient Descent (SVGD) is a widely used sampling algorithm that has been successfully applied in several areas of Machine Learning. SVGD operates by iteratively moving a set of interacting particles (which represent the samples) to approximate the target distribution. Despite recent studies on the complexity of SVGD and its variants, their long-time asymptotic behavior (i.e., after numerous iterations ) is still not understood in the finite number of particles regime. We study the long-time asymptotic behavior of a noisy variant of SVGD. First, we establish that the limit set of noisy SVGD for large is well-defined. We then characterize this limit set, showing that it approaches the target distribution as increases. In particular, noisy SVGD provably avoids the variance collapse observed for SVGD. Our approach involves demonstrating that the trajectories of noisy SVGD closely resemble those described by a McKean-Vlasov process.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑