arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1609.00368stat.MLcs.DSmath.STstat.TH
收藏

EM的十个步骤足以满足两个高斯混合

Ten Steps of EM Suffice for Mixtures of Two Gaussians

Constantinos Daskalakis, Christos Tzamos, Manolis Zampetakis

摘要:

期望最大化(EM)算法是一种广泛使用的极大似然估计方法。对于估计高斯混合,其迭代可以被视为k-means聚类算法的软版本。尽管它的广泛使用和应用,基本上没有已知的收敛保证这种方法。我们提供了全局收敛保证的混合两个高斯已知的协方差阵。我们表明,人口版本的EM,该算法是获得无穷多个样本的混合物,几何收敛到正确的平均向量,并提供简单的,封闭形式的表达式的收敛速度。作为一个简单的例子,我们表明,在一维,10个步骤的期望最大化算法初始化在无穷大的结果小于1\%的误差估计的平均值。在有限样本制度中,我们表明,在随机初始化,$\tilde{O}(d/ε^2)$样本足以计算未知向量到$ε$内的Mahalanobis距离,其中$d$是尺寸。特别地,基于EM的估计器的误差率是$\tilde{O}\left(\sqrt{d \over n}\right)$,其中$n$是样本的数量,其最佳达到对数因子。

英文摘要:

The Expectation-Maximization (EM) algorithm is a widely used method for maximum likelihood estimation in models with latent variables. For estimating mixtures of Gaussians, its iteration can be viewed as a soft version of the k-means clustering algorithm. Despite its wide use and applications, there are essentially no known convergence guarantees for this method. We provide global convergence guarantees for mixtures of two Gaussians with known covariance matrices. We show that the population version of EM, where the algorithm is given access to infinitely many samples from the mixture, converges geometrically to the correct mean vectors, and provide simple, closed-form expressions for the convergence rate. As a simple illustration, we show that, in one dimension, ten steps of the EM algorithm initialized at infinity result in less than 1\% error estimation of the means. In the finite sample regime, we show that, under a random initialization, $\tilde{O}(d/ε^2)$ samples suffice to compute the unknown vectors to within $ε$ in Mahalanobis distance, where $d$ is the dimension. In particular, the error rate of the EM based estimator is $\tilde{O}\left(\sqrt{d \over n}\right)$ where $n$ is the number of samples, which is optimal up to logarithmic factors.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑