arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2009.12703cs.LGstat.ML
收藏

高斯混合模型无监督学习的自适应EM加速器

An Adaptive EM Accelerator for Unsupervised Learning of Gaussian Mixture Models

Los Alamos National Laboratory(洛斯阿拉莫斯国家实验室)

Truong Nguyen, Guangye Chen, Luis Chacon

摘要:

我们提出了一个安德森加速(AA)计划的自适应期望最大化(EM)算法的无监督学习有限混合模型从多变量数据(Figueiredo和Jain 2002)。该算法能够自主地确定混合组分的最佳数量,并且比其非加速版本更快地收敛到最优解。基于AA的算法的成功源于几个发展,而不是一个单一的突破(如果没有这些,我们的测试表明AA失败的灾难性)。首先,我们使用最近提出的单调性控制算法(Henderson和Varahdan 2019)确保似然函数的单调性(标准EM算法的关键特征),并通过一种新的单调性测试进行增强,开销很小。我们提出了灵活的策略AA保持高斯权重和协方差矩阵的正定性严格,并保存到观察数据集的二阶矩准确。最后,我们采用了一个K-均值聚类算法,使用间隙统计,以避免过度高估初始组件的数量,从而最大限度地提高性能。我们证明了该算法的准确性和效率与几个合成数据集,是已知数量的组件的高斯分布的混合物,以及从粒子在细胞模拟生成的数据集。我们的数值计算结果表明,当混合物组分的确切数量是已知的,并且在几个和多个数量级之间具有组分自适应性时,相对于非加速EM的加速可达60 X。

英文摘要:

We propose an Anderson Acceleration (AA) scheme for the adaptive Expectation-Maximization (EM) algorithm for unsupervised learning a finite mixture model from multivariate data (Figueiredo and Jain 2002). The proposed algorithm is able to determine the optimal number of mixture components autonomously, and converges to the optimal solution much faster than its non-accelerated version. The success of the AA-based algorithm stems from several developments rather than a single breakthrough (and without these, our tests demonstrate that AA fails catastrophically). To begin, we ensure the monotonicity of the likelihood function (a the key feature of the standard EM algorithm) with a recently proposed monotonicity-control algorithm (Henderson and Varahdan 2019), enhanced by a novel monotonicity test with little overhead. We propose nimble strategies for AA to preserve the positive definiteness of the Gaussian weights and covariance matrices strictly, and to conserve up to the second moments of the observed data set exactly. Finally, we employ a K-means clustering algorithm using the gap statistic to avoid excessively overestimating the initial number of components, thereby maximizing performance. We demonstrate the accuracy and efficiency of the algorithm with several synthetic data sets that are mixtures of Gaussians distributions of known number of components, as well as data sets generated from particle-in-cell simulations. Our numerical results demonstrate speed-ups with respect to non-accelerated EM of up to 60X when the exact number of mixture components is known, and between a few and more than an order of magnitude with component adaptivity.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑