arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.05574math.STstat.MLstat.TH
收藏

一般混合模型的可证明初始化与鲁棒聚类方法

A provable initialization and robust clustering method for general mixture models

Soham Jana, Jianqing Fan, Sanjeev Kulkarni

AI总结:

本文针对一般混合模型提出基于多元截尾均值质心估计的鲁棒初始化与聚类方法,在有限簇数和维度下可证明达到接近最优的误标号率,并能抵抗重尾误差和对抗离群点,同时证明Lloyd算法在多簇或重尾情形下次优。

摘要:

聚类是统计机器学习中处理异质数据的基本工具。最近的大多数结果主要关注数据围绕质心呈亚高斯误差分布时的最优误标号保证。然而,限制性的亚高斯模型在实践中往往不成立,因为许多实际应用在质心附近呈现重尾分布,或者可能遭受对抗攻击,因此需要具有鲁棒数据驱动初始化的鲁棒聚类方法。本文提出了初始化及后续聚类方法,在簇数和数据维度有限的一般混合模型下,可证明地保证接近最优的误标号性能。我们首先引入一种混合聚类技术,采用新颖的多元截尾均值型质心估计,在质心周围一般误差分布的弱初始化条件下给出误标号保证。我们推导出匹配的下界,其差别仅取决于簇数的因子。此外,即使在存在对抗离群点的情况下,我们的方法也能给出类似的误标号保证。当误差服从亚高斯分布时,我们的结果在有限维度下退化为亚高斯情形。为彻底解决该问题,我们还提出了新颖的数据驱动鲁棒初始化技术,并证明这些初始质心估计以趋于1的概率足够好,使后续聚类算法能够达到最优误标号率。进一步,我们证明Lloyd算法在两个以上簇时即使误差为高斯分布也是次优的,而在两个簇且误差分布具有重尾时同样次优。模拟数据和真实数据示例进一步支持了我们的鲁棒初始化过程和聚类算法。

英文摘要:

Clustering is a fundamental tool in statistical machine learning in the presence of heterogeneous data. Most recent results focus primarily on optimal mislabeling guarantees when data are distributed around centroids with sub-Gaussian errors. Yet, the restrictive sub-Gaussian model is often invalid in practice since various real-world applications exhibit heavy tail distributions around the centroids or suffer from possible adversarial attacks that call for robust clustering with a robust data-driven initialization. In this paper, we present initialization and subsequent clustering methods that provably guarantee near-optimal mislabeling for general mixture models when the number of clusters and data dimensions are finite. We first introduce a hybrid clustering technique with a novel multivariate trimmed mean type centroid estimate to produce mislabeling guarantees under a weak initialization condition for general error distributions around the centroids. A matching lower bound is derived, up to factors depending on the number of clusters. In addition, our approach also produces similar mislabeling guarantees even in the presence of adversarial outliers. Our results reduce to the sub-Gaussian case in finite dimensions when errors follow sub-Gaussian distributions. To solve the problem thoroughly, we also present novel data-driven robust initialization techniques and show that, with probabilities approaching one, these initial centroid estimates are sufficiently good for the subsequent clustering algorithm to achieve the optimal mislabeling rates. Furthermore, we demonstrate that the Lloyd algorithm is suboptimal for more than two clusters even when errors are Gaussian and for two clusters when error distributions have heavy tails. Both simulated data and real data examples further support our robust initialization procedure and clustering algorithm.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑