集群可以随其集群呈次线性扩展吗？Gestival和$k$ -平均的变分EM加速度
Can clustering scale sublinearly with its clusters? A variational EM acceleration of GMMs and $k$-means
摘要：
标准$k$的一次迭代意味着（即，Lloyd算法）或用于高斯混合模型的标准EM（GARCH）与聚类$C$、数据点$N$和数据维度$D$的数量成线性比例。在这项研究中，我们探讨是否一个迭代的$k$ -均值或EM的GREEN可以缩放次线性与$C$在运行时，同时提高聚类目标仍然有效。我们用于降低复杂度的工具是变分EM，它通常用于训练具有指数级隐藏状态的生成模型。在这里，我们应用新的理论成果截断变分EM，使易于处理的聚类算法更有效。基本思想是使用部分变分E步骤，将完整E步骤所需的$\mathcal{O}(NCD)$的线性复杂度降低到亚线性复杂度。我们的主要观察是，对$C$的线性依赖性可以减少到一个小得多的参数$G$，它涉及到集群邻域关系的依赖性。我们专注于两个版本的部分变分EM聚类：变分GMM，缩放与$\mathcal{O}(NG^2D)$，和变分$k$ -手段，缩放与$\mathcal{O}(NGD)$每次迭代。实验结果表明，这些算法仍然需要相当数量的迭代，以提高聚类目标相同的值为$k$ -手段。对于具有许多集群的数据，我们因此观察到净计算需求减少了两到三个数量级。更一般地说，我们的研究结果提供了大量的经验证据，有利于集群规模与$C$次线性。
英文摘要：
One iteration of standard $k$-means (i.e., Lloyd's algorithm) or standard EM for Gaussian mixture models (GMMs) scales linearly with the number of clusters $C$, data points $N$, and data dimensionality $D$. In this study, we explore whether one iteration of $k$-means or EM for GMMs can scale sublinearly with $C$ at run-time, while improving the clustering objective remains effective. The tool we apply for complexity reduction is variational EM, which is typically used to make training of generative models with exponentially many hidden states tractable. Here, we apply novel theoretical results on truncated variational EM to make tractable clustering algorithms more efficient. The basic idea is to use a partial variational E-step which reduces the linear complexity of $\mathcal{O}(NCD)$ required for a full E-step to a sublinear complexity. Our main observation is that the linear dependency on $C$ can be reduced to a dependency on a much smaller parameter $G$ which relates to cluster neighborhood relations. We focus on two versions of partial variational EM for clustering: variational GMM, scaling with $\mathcal{O}(NG^2D)$, and variational $k$-means, scaling with $\mathcal{O}(NGD)$ per iteration. Empirical results show that these algorithms still require comparable numbers of iterations to improve the clustering objective to same values as $k$-means. For data with many clusters, we consequently observe reductions of net computational demands between two and three orders of magnitude. More generally, our results provide substantial empirical evidence in favor of clustering to scale sublinearly with $C$.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
