在MapReduce和流式处理中解决$k$中心聚类(带离群值),几乎与顺序一样精确
Solving $k$-center Clustering (with Outliers) in MapReduce and Streaming, almost as Accurately as Sequentially
摘要:
基于中心的聚类是数据分析的基础,对于大型数据集来说变得非常具有挑战性。在本文中,我们专注于流行的$k$ -中心变量,给定一组$S$的点从一些度量空间和一个参数$k<|S|$,需要确定一个子集的$k$中心在$S$最小化的最大距离的任何点的$S$从其最近的中心。一个更一般的公式,介绍了处理噪声数据集,具有进一步的参数$z$,并允许高达$z$点的$S$(离群值)被忽略时,计算最大距离的中心。我们提出了基于核心集的2轮MapReduce算法的上述两个公式的问题,和1-通过流算法的情况下,离群值。对于任何固定的$ε>0$,算法产生的解决方案,其近似比是一个纯粹的加法项$ε$远离那些可实现的最知名的多项式时间顺序算法,结果大大提高后的国家的最先进的。我们的算法是相当简单,并适应于数据集的内在复杂性,捕获的双维度$D$的度量空间。具体来说,我们的分析表明,该算法变得非常空间效率的小(常数)$D$的重要情况下。这些理论结果得到了一组在真实世界和超过10亿个点的合成数据集上进行的实验的补充,这些实验表明,我们的算法在具有出色的可扩展性的同时,可以产生比现有算法更高质量的解决方案,并且它们还可以比现有算法更快地顺序实现。
英文摘要:
Center-based clustering is a fundamental primitive for data analysis and becomes very challenging for large datasets. In this paper, we focus on the popular $k$-center variant which, given a set $S$ of points from some metric space and a parameter $k<|S|$, requires to identify a subset of $k$ centers in $S$ minimizing the maximum distance of any point of $S$ from its closest center. A more general formulation, introduced to deal with noisy datasets, features a further parameter $z$ and allows up to $z$ points of $S$ (outliers) to be disregarded when computing the maximum distance from the centers. We present coreset-based 2-round MapReduce algorithms for the above two formulations of the problem, and a 1-pass Streaming algorithm for the case with outliers. For any fixed $ε>0$, the algorithms yield solutions whose approximation ratios are a mere additive term $ε$ away from those achievable by the best known polynomial-time sequential algorithms, a result that substantially improves upon the state of the art. Our algorithms are rather simple and adapt to the intrinsic complexity of the dataset, captured by the doubling dimension $D$ of the metric space. Specifically, our analysis shows that the algorithms become very space-efficient for the important case of small (constant) $D$. These theoretical results are complemented with a set of experiments on real-world and synthetic datasets of up to over a billion points, which show that our algorithms yield better quality solutions over the state of the art while featuring excellent scalability, and that they also lend themselves to sequential implementations much faster than existing ones.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。