用于采样的梯度流:平均场模型、高斯近似与仿射不变性
Gradient Flows for Sampling: Mean-Field Models, Gaussian Approximations and Affine Invariance
AI总结:
本文研究带未知归一化常数的采样问题,通过选择Kullback-Leibler散度能量和Fisher-Rao、Wasserstein、Stein度量构造梯度流,引入并实现仿射不变性,并证明高斯近似与矩闭合一致且具有更好的长时间收敛性质。
摘要:
对具有未知归一化常数的概率分布进行采样是计算科学与工程中的一个基本问题。该任务可以被表述为在所有概率测度上的优化问题,并且初始分布可以通过梯度流动态地演化到期望的极小化子。其规律由概率测度空间中的梯度流控制的平均场模型也可以被识别出来;这些平均场模型的粒子近似构成了算法的基础。梯度流方法也是变分推断算法的基础,在变分推断中,优化是在诸如高斯分布之类的参数化概率分布族上进行的,并且底层的梯度流被限制在该参数化族上。通过为梯度流选择不同的能量泛函和度量,会产生具有不同收敛性质的不同算法。在本文中,我们专注于Kullback-Leibler散度,并首先证明,在相差一个尺度因子的意义下,它具有如下唯一性质:由这一能量选择所产生的梯度流不依赖于归一化常数。对于度量,我们聚焦于Fisher-Rao、Wasserstein和Stein度量的变体;我们为梯度流及其相应的平均场模型引入仿射不变性性质,确定给定度量是否导致仿射不变性,并在其不具备仿射不变性时对其进行修改以使其具有仿射不变性。我们研究在概率密度空间和高斯空间中产生的梯度流。高斯空间中的流可以被理解为该流的高斯近似。我们证明基于该度量的高斯近似与通过矩闭合得到的高斯近似是一致的,建立它们之间的联系,并研究它们的长时间收敛性质,从而展示仿射不变性的优势。
英文摘要:
Sampling a probability distribution with an unknown normalization constant is a fundamental problem in computational science and engineering. This task may be cast as an optimization problem over all probability measures, and an initial distribution can be evolved to the desired minimizer dynamically via gradient flows. Mean-field models, whose law is governed by the gradient flow in the space of probability measures, may also be identified; particle approximations of these mean-field models form the basis of algorithms. The gradient flow approach is also the basis of algorithms for variational inference, in which the optimization is performed over a parameterized family of probability distributions such as Gaussians, and the underlying gradient flow is restricted to the parameterized family. By choosing different energy functionals and metrics for the gradient flow, different algorithms with different convergence properties arise. In this paper, we concentrate on the Kullback-Leibler divergence after showing that, up to scaling, it has the unique property that the gradient flows resulting from this choice of energy do not depend on the normalization constant. For the metrics, we focus on variants of the Fisher-Rao, Wasserstein, and Stein metrics; we introduce the affine invariance property for gradient flows, and their corresponding mean-field models, determine whether a given metric leads to affine invariance, and modify it to make it affine invariant if it does not. We study the resulting gradient flows in both probability density space and Gaussian space. The flow in the Gaussian space may be understood as a Gaussian approximation of the flow. We demonstrate that the Gaussian approximation based on the metric and through moment closure coincide, establish connections between them, and study their long-time convergence properties showing the advantages of affine invariance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。