arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1801.04289stat.MLcs.DCcs.LG
收藏

异步随机变分推断

Asynchronous Stochastic Variational Inference

Bournemouth University(伯恩茅斯大学);IMT Lille Douai(IMT里尔杜埃)

Saad Mohamad, Abdelhamid Bouchachia, Moamar Sayed-Mouchaweh

摘要:

随机变分推断(SVI)采用随机最优化将贝叶斯计算扩展到海量数据。由于SVI的核心是一种基于随机梯度的算法,因此可以利用水平并行来进行更大规模的推理。我们提出了一个无锁的并行实现SVI,它允许分布式计算在多个奴隶在异步风格。我们表明,我们的实现导致线性加速,同时保证渐近遍历收敛速度($O(1/\sqrt(T)$)给定的奴隶的数量是有界的$\sqrt(T)$($T$是迭代的总次数)。该实现是在高性能计算(HPC)环境中使用Python的消息传递接口(MPI)(MPI4py)完成的。大量的实证评估表明,我们的并行SVI是无损的,表现良好,其对应的串行SVI与线性加速。

英文摘要:

Stochastic variational inference (SVI) employs stochastic optimization to scale up Bayesian computation to massive data. Since SVI is at its core a stochastic gradient-based algorithm, horizontal parallelism can be harnessed to allow larger scale inference. We propose a lock-free parallel implementation for SVI which allows distributed computations over multiple slaves in an asynchronous style. We show that our implementation leads to linear speed-up while guaranteeing an asymptotic ergodic convergence rate $O(1/\sqrt(T)$ ) given that the number of slaves is bounded by $\sqrt(T)$ ($T$ is the total number of iterations). The implementation is done in a high-performance computing (HPC) environment using message passing interface (MPI) for python (MPI4py). The extensive empirical evaluation shows that our parallel SVI is lossless, performing comparably well to its counterpart serial SVI with linear speed-up.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑