arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1805.06951cs.LGstat.ML
收藏

无块并行推理的森林混合界

A Forest Mixture Bound for Block-Free Parallel Inference

University of Southern California(南加州大学)

Neal Lawton, Aram Galstyan, Greg Ver Steeg

摘要:

坐标上升变分推理是概率模型推理的一种重要算法,但由于每次只更新一个变量,速度较慢。块坐标方法通过并行更新变量块来更快地执行推理。然而,这些算法的速度和稳定性取决于如何将变量划分成块。在本文中,我们给出了一个稳定的并行算法推理深指数族,不需要将变量划分成块。我们通过一个新的目标来降低ELBO的边界,我们称之为森林混合边界(FM边界),它将隐藏层中变量的推理问题分开。我们将其应用到简单的情况下,当所有的随机变量都是高斯和经验表明,该算法收敛速度更快的模型,本质上更像森林。

英文摘要:

Coordinate ascent variational inference is an important algorithm for inference in probabilistic models, but it is slow because it updates only a single variable at a time. Block coordinate methods perform inference faster by updating blocks of variables in parallel. However, the speed and stability of these algorithms depends on how the variables are partitioned into blocks. In this paper, we give a stable parallel algorithm for inference in deep exponential families that doesn't require the variables to be partitioned into blocks. We achieve this by lower bounding the ELBO by a new objective we call the forest mixture bound (FM bound) that separates the inference problem for variables within a hidden layer. We apply this to the simple case when all random variables are Gaussian and show empirically that the algorithm converges faster for models that are inherently more forest-like.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑