我如何学会停止担心并爱上维度的诅咒：对高维空间中聚类验证的评估
How I learned to stop worrying and love the curse of dimensionality: an appraisal of cluster validation in high-dimensional spaces
摘要：
众所周知，欧几里得范数无法可靠地区分高维空间中的近点和远点。这种距离集中现象表现在各种数据分布中，具有iid或相关特征，包括集中分布和集群数据。因此，基于欧几里得最近邻的无监督学习和更一般的面向邻近的数据采掘任务（如聚类）可能会受到高维应用距离集中的不利影响。虽然已经做了大量的工作，开发具有可靠的高维性能的聚类算法，聚类验证的问题-确定在数据集中的聚类的自然数-还没有仔细检查在高维问题。在这项工作中，我们调查如何常见的欧氏范数为基础的集群有效性指标规模的灵敏度与尺寸的各种合成数据计划，包括良好分离和嘈杂的集群，并发现绝大多数的指标提高或稳定的灵敏度在高维度。因此，这类相当通用的数据方案消除了维数灾难。
英文摘要：
The failure of the Euclidean norm to reliably distinguish between nearby and distant points in high dimensional space is well-known. This phenomenon of distance concentration manifests in a variety of data distributions, with iid or correlated features, including centrally-distributed and clustered data. Unsupervised learning based on Euclidean nearest-neighbors and more general proximity-oriented data mining tasks like clustering, might therefore be adversely affected by distance concentration for high-dimensional applications. While considerable work has been done developing clustering algorithms with reliable high-dimensional performance, the problem of cluster validation--of determining the natural number of clusters in a dataset--has not been carefully examined in high-dimensional problems. In this work we investigate how the sensitivities of common Euclidean norm-based cluster validity indices scale with dimension for a variety of synthetic data schemes, including well-separated and noisy clusters, and find that the overwhelming majority of indices have improved or stable sensitivity in high dimensions. The curse of dimensionality is therefore dispelled for this class of fairly generic data schemes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
