arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2112.00945cs.LG
收藏

DPVI:一个基于动态权重粒子的变分推理框架

DPVI: A Dynamic-Weight Particle-Based Variational Inference Framework

College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);Information Science and Electronic Engineering, Zhejiang University(浙江大学信息与电子工程学院)

Chao Zhang, Zhijian Li, Hui Qian, Xin Du

摘要:

最近发展的基于粒子的变分推理(ParVI)方法通过迭代更新粒子的位置来驱动一组\emph{fixed-weight}粒子的经验分布朝向给定的目标分布$π$。然而,固定权重的限制极大地限制了经验分布的逼近能力,特别是当粒子数有限时。在本文中,我们提出了动态调整粒子的权重,根据Fisher-Rao反应流。我们开发了一个通用的动态权重基于粒子的变分推理(DPVI)框架,根据一种新的连续复合流,它同时进化粒子的位置和权重。我们发现,我们的复合流的平均场极限实际上是一个Wasserstein-Fisher-Rao梯度流的某些相异功能$\mathcal{F}$,这导致更快地减少$\mathcal{F}$比Wasserstein梯度流现有的固定重量ParVI。通过在我们的一般框架中使用不同的有限粒子近似,我们得到了几个有效的DPVI算法。实验结果表明,我们推导出的DPVI算法优于其固定权重的同行。

英文摘要:

The recently developed Particle-based Variational Inference (ParVI) methods drive the empirical distribution of a set of \emph{fixed-weight} particles towards a given target distribution $π$ by iteratively updating particles' positions. However, the fixed weight restriction greatly confines the empirical distribution's approximation ability, especially when the particle number is limited. In this paper, we propose to dynamically adjust particles' weights according to a Fisher-Rao reaction flow. We develop a general Dynamic-weight Particle-based Variational Inference (DPVI) framework according to a novel continuous composite flow, which evolves the positions and weights of particles simultaneously. We show that the mean-field limit of our composite flow is actually a Wasserstein-Fisher-Rao gradient flow of certain dissimilarity functional $\mathcal{F}$, which leads to a faster decrease of $\mathcal{F}$ than the Wasserstein gradient flow underlying existing fixed-weight ParVIs. By using different finite-particle approximations in our general framework, we derive several efficient DPVI algorithms. The empirical results demonstrate the superiority of our derived DPVI algorithms over their fixed-weight counterparts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑