arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2109.08969stat.COstat.MEstat.ML
收藏

针对海量数据设置的异步和分布式数据扩充

Asynchronous and Distributed Data Augmentation for Massive Data Settings

Jiayuan Zhou, Kshitij Khare, Sanvesh Srivastava

摘要:

数据增广(DA)算法由于其简单性而被广泛用于贝叶斯推断。然而,在大规模数据设置中,DA算法非常慢,因为它们在任何迭代中都要通过完整的数据,尽管有优势,但对它们的使用施加了严重的限制。为了解决这个问题,我们开发了一个框架,用于扩展任何DA,利用异步和分布式计算。扩展的DA算法由参数$r \in (0, 1)$索引,并且被称为异步和分布式(AD)DA,其中原始DA作为其父DA。任何ADDA都是通过将完整数据划分为$k$较小的不相交子集并将它们存储在$k$进程(可以是机器或处理器)上开始的。ADDA的每次迭代仅以某个正概率增加$k$数据子集的$r$ -分数,并且保持增加的数据的剩余$(1-r)$ -分数不变。参数绘制使用新的$r$分数和旧的增强数据的$(1-r)$分数获得。对于$k$和$r$的许多选择,ADDA的部分更新导致在大规模数据设置中比父DA显著加速,并且当$r=1$时,它减少到其父DA的分布式版本。我们证明了ADDA马尔可夫链是Harris遍历的期望平稳分布在温和的条件下的父DA算法。我们在三个有代表性的例子中展示了ADDA的数值优势,对应于应用中遇到的不同类型的海量数据设置。在所有这些示例中,对于$k$和$r$的所有选择,我们的DA泛化明显快于其父DA算法。我们还建立了几何遍历的ADDA马尔可夫链的所有三个例子,这反过来又产生渐近有效的标准误所需的后验量的估计。

英文摘要:

Data augmentation (DA) algorithms are widely used for Bayesian inference due to their simplicity. In massive data settings, however, DA algorithms are prohibitively slow because they pass through the full data in any iteration, imposing serious restrictions on their usage despite the advantages. Addressing this problem, we develop a framework for extending any DA that exploits asynchronous and distributed computing. The extended DA algorithm is indexed by a parameter $r \in (0, 1)$ and is called Asynchronous and Distributed (AD) DA with the original DA as its parent. Any ADDA starts by dividing the full data into $k$ smaller disjoint subsets and storing them on $k$ processes, which could be machines or processors. Every iteration of ADDA augments only an $r$-fraction of the $k$ data subsets with some positive probability and leaves the remaining $(1-r)$-fraction of the augmented data unchanged. The parameter draws are obtained using the $r$-fraction of new and $(1-r)$-fraction of old augmented data. For many choices of $k$ and $r$, the fractional updates of ADDA lead to a significant speed-up over the parent DA in massive data settings, and it reduces to the distributed version of its parent DA when $r=1$. We show that the ADDA Markov chain is Harris ergodic with the desired stationary distribution under mild conditions on the parent DA algorithm. We demonstrate the numerical advantages of the ADDA in three representative examples corresponding to different kinds of massive data settings encountered in applications. In all these examples, our DA generalization is significantly faster than its parent DA algorithm for all the choices of $k$ and $r$. We also establish geometric ergodicity of the ADDA Markov chain for all three examples, which in turn yields asymptotically valid standard errors for estimates of desired posterior quantities.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑