Sampling via Gradient Flows in the Space of Probability Measures
California Institute of Technology(加州理工学院);Courant Institute, New York University(纽约大学柯朗数学研究所);Beijing International Center for Mathematical Research, Peking University(北京大学北京国际数学研究中心);University of Pennsylvania(宾夕法尼亚大学);Universität Potsdam(波茨坦大学)
Yifan Chen, Daniel Zhengyu Huang, Jiaoyang Huang, Sebastian Reich, Andrew M Stuart
Sampling a target probability distribution with an unknown normalization constant is a fundamental challenge in computational science and engineering. Recent work shows that algorithms derived by considering gradient flows in the space of probability measures open up new avenues for algorithm development. This paper makes three contributions to this sampling approach by scrutinizing the design components of such gradient flows. Any instantiation of a gradient flow for sampling needs an energy functional and a metric to determine the flow, as well as numerical approximations of the flow to derive algorithms. Our first contribution is to show that the Kullback-Leibler divergence, as an energy functional, has the unique property (among all f-divergences) that gradient flows resulting from it do not depend on the normalization constant of the target distribution. Our second contribution is to study the choice of metric from the perspective of invariance. The Fisher-Rao metric is known as the unique choice (up to scaling) that is diffeomorphism invariant. As a computationally tractable alternative, we introduce a relaxed, affine invariance property for the metrics and gradient flows. In particular, we construct various affine invariant Wasserstein and Stein gradient flows. Affine invariant gradient flows are shown to behave more favorably than their non-affine-invariant counterparts when sampling highly anisotropic distributions, in theory and by using particle methods. Our third contribution is to study, and develop efficient algorithms based on Gaussian approximations of the gradient flows; this leads to an alternative to particle methods. We establish connections between various Gaussian approximate gradient flows, discuss their relation to gradient methods arising from parametric variational inference, and study their convergence properties both theoretically and numerically.
When Langevin Monte Carlo Meets Randomization: New Sampling Algorithms with Non-asymptotic Error Bounds beyond Log-Concavity and Gradient Lipschitzness
School of Mathematics and Statistics, HNP-LAMA, Central South University(数学与统计学院,HNP-LAMA,中南大学)
Improving Sampling Accuracy of Stochastic Gradient MCMC Methods via Non-uniform Subsampling of Gradients
Georgia Institute of Technology(佐治亚理工学院);Google Inc.(谷歌公司);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))
许多马尔可夫链蒙特卡罗(MCMC)方法利用目标分布的势函数的梯度信息来有效地探索样本空间。然而,计算梯度对于大规模应用(诸如当代机器学习中的那些应用)而言通常在计算上是昂贵的。随机梯度(SG-)MCMC方法通常通过均匀子采样的数据点通过随机梯度来近似梯度,并且实现了提高的计算效率,但是以引入采样误差为代价。我们提出了一种非均匀子采样方案,以提高采样精度。提出的指数加权随机梯度(EWSG)的设计,使一个非均匀SG-MCMC方法模仿的批量梯度MCMC方法的统计行为,因此,由于SG近似的不准确性减少。EWSG不同于经典的方差缩减(VR)技术,因为它关注的是整个分布而不仅仅是方差;尽管如此,它的局部方差也被证明了。EWSG也可以被看作是重要性采样思想的扩展,成功的随机梯度为基础的优化,采样任务。在EWSG的实际实现中,非均匀子采样通过数据索引上的Metropolis-Hastings链有效地执行,该链与MCMC算法耦合。数值实验不仅证明了EWSG的有效性,而且还指导超参数的选择,并验证我们的\emph{non-asymptotic global error bound},尽管在实现中的近似。值得注意的是,虽然统计精度得到了提高,但收敛速度可以与统一版本相媲美,这使得EWSG成为VR的实用替代方案(但EWSG和VR也可以组合)。
From Importance Sampling to Doubly Robust Policy Gradient
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
我们表明,对政策的策略梯度(PG)和它的方差减少变量可以通过采取有限差分的功能评估提供的估计从重要性采样(IS)家庭的政策外评估(OPE)。从双重稳健(DR)估计量(Jiang & Li,2016)开始,我们提供了一种非常通用和灵活的PG形式的简单推导,其中包含了最先进的方差缩减技术(Cheng et al.,2019年)作为其特例,并立即暗示了现有文献所忽视的进一步降低方差的机会。我们分析了新的DR-PG估计的方差,将其与现有的方法以及策略梯度的Cramer-Rao下界进行了比较,并实证表明了其有效性。
Semi-Implicit Functional Gradient Flow for Efficient Sampling
School of Mathematical Sciences, Peking University(北京大学数学科学学院);Department of Industrial Engineering and Operations Research, University of California, Berkeley(加州大学伯克利分校工业工程与运筹学系)
Further and stronger analogy between sampling and optimization: Langevin Monte Carlo and gradient descent
在本文中,我们重新审视最近建立的理论保证的Langevin Monte Carlo算法的采样从光滑和(强)对数凹密度的收敛性。我们改进了现有的结果时,收敛测量Wasserstein距离,并提供进一步的见解非常紧密的关系,一方面,朗之万蒙特卡罗采样,另一方面,梯度下降优化。最后,我们还建立了一个版本的朗之万蒙特卡罗算法,是基于噪声的梯度评估的收敛性保证。
Branching Stein Variational Gradient Descent for sampling multimodal distributions
) Kyoto University, Disaster Prevention Research Institute, Japan;) Centro de Investigaci\'on en Matem\'aticas A.C., Department of Probability;) ETH Zurich, Department of Mathematics, Switzerland
On the Convergence of Wasserstein Gradient Descent for Sampling
VNU University of Science, Hanoi, Vietnam(越南国立大学科学大学);Vin University, Hanoi, Vietnam(越南文大学);The University of Sydney Business School, Australia(澳大利亚悉尼大学商学院)
Sampling in Constrained Domains with Orthogonal-Space Variational Gradient Descent
Purdue University(普渡大学);University of Texas at Austin(德克萨斯大学奥斯汀分校);National University of Singapore(新加坡国立大学)
抽样方法是一种重要的推理和学习技术,通常是针对无约束领域设计的。然而,在机器学习问题中,约束是普遍存在的,例如关于安全性、公平性、鲁棒性和许多其他属性的约束,这些约束必须满足才能将采样结果应用于现实生活中的应用。强制执行这些约束通常会导致隐式定义的流形,使得具有约束的有效采样非常具有挑战性。在本文中,我们提出了一个新的变分框架与设计的正交空间梯度流(O-梯度)采样流形$\mathcal{G}_0$定义的一般等式约束。O-Gradient将梯度分解为两部分:一部分减小到$\mathcal{G}_0$的距离,另一部分减小正交空间中的KL散度。虽然大多数现有的流形采样方法需要在$\mathcal{G}_0$上进行初始化,但O-梯度不需要这样的先验知识。证明了在较弱的条件下,O-梯度以速率$\widetilde{O}(1/\text{the number of iterations})$收敛于目标约束分布.我们的证明依赖于一个新的条件测度的Stein特征,这可能是独立的利益。我们通过Langevin动力学和Stein变分梯度下降实现O-梯度,并在各种实验中证明了其有效性,包括贝叶斯深度神经网络。
Robust Stochastic Gradient Posterior Sampling with Lattice Based Discretisation
Institute of Physics, University of Amsterdam, Netherlands(阿姆斯特丹大学物理研究所);Institute for Mathematics, Academia Sinica, Taiwan(台湾);Korteweg-de Vries Institute for Mathematics, University of Amsterdam, Netherlands(阿姆斯特丹大学克罗特-德-维尔斯数学研究所);Department of Computer Science, University of Oxford, United Kingdom(牛津大学计算机科学系);Normal Computing Corporation, New York, New York, USA(正常计算公司);Amsterdam Machine Learning Lab, University of Amsterdam, Netherlands(阿姆斯特丹机器学习实验室);Department of Physics, National Taiwan University, Taiwan(台湾国立台湾大学物理系)
Improving Gradient-guided Nested Sampling for Posterior Inference
Mila – Québec Artificial Intelligence Institute(米拉-魁北克人工智能研究所);Université de Montréal(蒙特利尔大学);Ciela Institute(谢拉研究所);CCA – Flatiron Institute(计算天体物理学中心-弗拉蒂伦研究所);Cavendish Laboratory(卡文迪什实验室);Kavli Institute for Cosmology(卡夫利宇宙学研究所);Gonville and Caius College, University of Cambridge(剑桥大学冈维尔与凯斯学院);Canadian Institute for Advanced Research (CIFAR)(加拿大高等研究院)
Warwick Manufacturing Group (WMG), University of Warwick(华威大学华威制造集团(WMG));University of Warwick(华威大学);United Kingdom(英国(注意:英国是国家,非机构,这里可能是原文本中地址信息,按要求不提取))