一种面向海量数据的异步分布式期望最大化算法--DEM算法
An Asynchronous Distributed Expectation Maximization Algorithm For Massive Data: The DEM Algorithm
摘要：
期望最大化（EM）算法家族提供了一种通用方法来拟合大型复杂数据的灵活模型。EM型算法的期望（E）步骤在大规模数据应用中是耗时的，因为它需要多次通过完整的数据。我们解决这个问题，提出了一个异步和分布式广义的EM称为分布式EM（DEM）。使用DEM，现有的EM型算法很容易扩展到大规模的数据设置，利用分治技术和广泛可用的计算能力，如网格计算。DEM算法保留两组计算过程，称为\emph{workers}和\emph{managers}，分别用于执行E步骤和最大化步骤（M步骤）。样本被随机划分为大量不相交的子集，并存储在工作进程上。DEM算法的E步是在所有工作者上并行执行的，每个工作者在本地E步结束时将其结果传达给管理者。管理器在收到来自一部分工作者的$γ$结果后执行M步骤，其中$γ$是$(0, 1]$中的固定常数。由DEM算法生成的参数估计序列保留EM的吸引人的特性：参数估计序列收敛到局部模式和线性全局收敛速度。在不同的模拟集中在线性混合效应模型，DEM算法是显着快于竞争EM型算法，同时具有类似的精度。DEM算法在包含1000万个评级的电影评级数据库上保持其优越的经验性能。
英文摘要：
The family of Expectation-Maximization (EM) algorithms provides a general approach to fitting flexible models for large and complex data. The expectation (E) step of EM-type algorithms is time-consuming in massive data applications because it requires multiple passes through the full data. We address this problem by proposing an asynchronous and distributed generalization of the EM called the Distributed EM (DEM). Using DEM, existing EM-type algorithms are easily extended to massive data settings by exploiting the divide-and-conquer technique and widely available computing power, such as grid computing. The DEM algorithm reserves two groups of computing processes called \emph{workers} and \emph{managers} for performing the E step and the maximization step (M step), respectively. The samples are randomly partitioned into a large number of disjoint subsets and are stored on the worker processes. The E step of DEM algorithm is performed in parallel on all the workers, and every worker communicates its results to the managers at the end of local E step. The managers perform the M step after they have received results from a $γ$-fraction of the workers, where $γ$ is a fixed constant in $(0, 1]$. The sequence of parameter estimates generated by the DEM algorithm retains the attractive properties of EM: convergence of the sequence of parameter estimates to a local mode and linear global rate of convergence. Across diverse simulations focused on linear mixed-effects models, the DEM algorithm is significantly faster than competing EM-type algorithms while having a similar accuracy. The DEM algorithm maintains its superior empirical performance on a movie ratings database consisting of 10 million ratings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
