arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.00546math.STstat.TH
收藏

关于缺失数据 k-means 聚类的一些注记

Some notes on the $k$-means clustering for missing data

Yoshikazu Terada, Xin Guan

AI总结:

本文从理论上证明了缺失数据 k-means 扩展方法 k-POD 即使在完全随机缺失下也不一致,其期望损失可表示为部分变量 k-means 损失的加权和,并指出其在高维低缺失率场景下仍具适用性。

摘要:

经典 k-means 聚类要求完整的数据矩阵,不能有缺失项。作为缺失数据 k-means 聚类的自然扩展,k-POD 聚类已被提出,它在 k-means 聚类中忽略缺失项。本文证明了即使在完全随机缺失机制下,k-POD 聚类也不具有一致性。更具体地说,k-POD 聚类的期望损失可以表示为部分变量下期望 k-means 损失的加权和。因此,当样本量趋于无穷时,k-POD 聚类收敛到与 k-means 聚类不同的聚类结果。这一结果表明,尽管 k-means 聚类表现良好,k-POD 聚类可能无法捕捉隐藏的聚类结构。另一方面,对于高维数据,当每个变量的缺失率较低时,k-POD 聚类可能是一个合适的选择。

英文摘要:

The classical $k$-means clustering requires a complete data matrix without missing entries. As a natural extension of the $k$-means clustering for missing data, the $k$-POD clustering has been proposed, which ignores the missing entries in the $k$-means clustering. This paper shows the inconsistency of the $k$-POD clustering even under the missing completely at random mechanism. More specifically, the expected loss of the $k$-POD clustering can be represented as the weighted sum of the expected $k$-means losses with parts of variables. Thus, the $k$-POD clustering converges to the different clustering from the $k$-means clustering as the sample size goes to infinity. This result indicates that although the $k$-means clustering works well, the $k$-POD clustering may fail to capture the hidden cluster structure. On the other hand, for high-dimensional data, the $k$-POD clustering could be a suitable choice when the missing rate in each variable is low.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑