arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1806.07137stat.COcs.LGstat.ML
收藏

基于概率单纯形的大规模随机抽样

Large-Scale Stochastic Sampling from the Probability Simplex

Lancaster University(兰卡斯特大学);University of Washington(华盛顿大学)

Jack Baker, Paul Fearnhead, Emily B Fox, Christopher Nemeth

摘要:

随机梯度马尔可夫链蒙特卡罗(SGMCMC)已成为一种流行的可扩展贝叶斯推断方法。这些方法基于对连续时间过程的离散时间近似进行采样,例如朗之万扩散。当应用于定义在约束空间上的分布时,当我们靠近空间的边界时,时间离散化误差可以占主导地位。我们证明,正因为如此,目前的SGMCMC方法的单形斗争与稀疏的单形空间,当许多组件接近零。不幸的是,许多流行的大规模贝叶斯模型,如网络或主题模型,需要在稀疏单纯形空间上进行推理。为了避免这种离散化误差所造成的偏差,我们提出了随机Cox-Ingersoll-Ross过程(SCIR),它消除了所有的离散化误差,我们证明了从SCIR过程的样本是渐近无偏的。我们将讨论如何将这一想法扩展到其他受约束的空间。使用的SCIR过程中的SGMCMC算法的主题模型和Dirichlet过程的混合物模型比现有的SGMCMC方法提供了更好的性能。

英文摘要:

Stochastic gradient Markov chain Monte Carlo (SGMCMC) has become a popular method for scalable Bayesian inference. These methods are based on sampling a discrete-time approximation to a continuous time process, such as the Langevin diffusion. When applied to distributions defined on a constrained space the time-discretization error can dominate when we are near the boundary of the space. We demonstrate that because of this, current SGMCMC methods for the simplex struggle with sparse simplex spaces; when many of the components are close to zero. Unfortunately, many popular large-scale Bayesian models, such as network or topic models, require inference on sparse simplex spaces. To avoid the biases caused by this discretization error, we propose the stochastic Cox-Ingersoll-Ross process (SCIR), which removes all discretization error and we prove that samples from the SCIR process are asymptotically unbiased. We discuss how this idea can be extended to target other constrained spaces. Use of the SCIR process within a SGMCMC algorithm is shown to give substantially better performance for a topic model and a Dirichlet process mixture model than existing SGMCMC approaches.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑