arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.03597stat.MLcs.LGcs.NAmath.DSmath.NA
收藏

概率测度空间中基于梯度流的采样方法

Sampling via Gradient Flows in the Space of Probability Measures

California Institute of Technology(加州理工学院);Courant Institute, New York University(纽约大学柯朗数学研究所);Beijing International Center for Mathematical Research, Peking University(北京大学北京国际数学研究中心);University of Pennsylvania(宾夕法尼亚大学);Universität Potsdam(波茨坦大学)

Yifan Chen, Daniel Zhengyu Huang, Jiaoyang Huang, Sebastian Reich, Andrew M Stuart

AI总结:

本文研究概率测度空间梯度流采样方法,证明KL散度的归一化常数无关性,提出仿射不变梯度流并验证其在各向异性分布采样上的优势,还开发了高斯近似的高效采样算法。

摘要:

对归一化常数未知的目标概率分布进行采样是计算科学与工程领域的一项基础挑战。近期研究表明,通过考量概率测度空间中的梯度流推导得到的算法为算法开发开辟了新路径。本文通过细致分析此类梯度流的设计组成部分,为该采样方法做出三项贡献。任何用于采样的梯度流实例都需要一个能量泛函和一种度量来确定流的形式,还需要对流进行数值近似以推导得到算法。我们的第一项贡献是证明,在所有f散度中,作为能量泛函的Kullback-Leibler散度具备独特性质:由其推导得到的梯度流不依赖于目标分布的归一化常数。第二项贡献是从不变性的角度研究度量的选择。已知Fisher-Rao度量是(在缩放意义下)唯一具备微分同胚不变性的选择。作为一种计算上易处理的替代方案,我们为度量和梯度流提出了一种松弛的仿射不变性性质。具体而言,我们构建了多种具备仿射不变性的Wasserstein梯度流和Stein梯度流。理论分析及粒子方法实验均表明,在对高度各向异性的分布进行采样时,仿射不变梯度流的表现优于非仿射不变的对应梯度流。第三项贡献是研究并开发基于梯度流高斯近似的高效算法,这为粒子方法提供了一种替代方案。我们建立了多种高斯近似梯度流之间的关联,讨论了它们与参数化变分推理衍生的梯度方法之间的关系,并从理论和数值两方面研究了其收敛特性。

英文摘要:

Sampling a target probability distribution with an unknown normalization constant is a fundamental challenge in computational science and engineering. Recent work shows that algorithms derived by considering gradient flows in the space of probability measures open up new avenues for algorithm development. This paper makes three contributions to this sampling approach by scrutinizing the design components of such gradient flows. Any instantiation of a gradient flow for sampling needs an energy functional and a metric to determine the flow, as well as numerical approximations of the flow to derive algorithms. Our first contribution is to show that the Kullback-Leibler divergence, as an energy functional, has the unique property (among all f-divergences) that gradient flows resulting from it do not depend on the normalization constant of the target distribution. Our second contribution is to study the choice of metric from the perspective of invariance. The Fisher-Rao metric is known as the unique choice (up to scaling) that is diffeomorphism invariant. As a computationally tractable alternative, we introduce a relaxed, affine invariance property for the metrics and gradient flows. In particular, we construct various affine invariant Wasserstein and Stein gradient flows. Affine invariant gradient flows are shown to behave more favorably than their non-affine-invariant counterparts when sampling highly anisotropic distributions, in theory and by using particle methods. Our third contribution is to study, and develop efficient algorithms based on Gaussian approximations of the gradient flows; this leads to an alternative to particle methods. We establish connections between various Gaussian approximate gradient flows, discuss their relation to gradient methods arising from parametric variational inference, and study their convergence properties both theoretically and numerically.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑