Clustering is an essential task to unsupervised learning. It tries to automatically separate instances into coherent subsets. As one of the most well-known clustering algorithms, k-means assigns sample points at the boundary to a unique cluster, while it does not utilize the information of sample distribution or density. Comparably, it would potentially be more beneficial to consider the probability of each sample in a possible cluster. To this end, this paper generalizes k-means to model the distribution of clusters. Our novel clustering algorithm thus models the distributions of distances to centroids over a threshold by Generalized Pareto Distribution (GPD) in Extreme Value Theory (EVT). Notably, we propose the concept of centroid margin distance, use GPD to establish a probability model for each cluster, and perform a clustering algorithm based on the covering probability function derived from GPD. Such a GPD k-means thus enables the clustering algorithm from the probabilistic perspective. Correspondingly, we also introduce a naive baseline, dubbed as Generalized Extreme Value (GEV) k-means. GEV fits the distribution of the block maxima. In contrast, the GPD fits the distribution of distance to the centroid exceeding a sufficiently large threshold, leading to a more stable performance of GPD k-means. Notably, GEV k-means can also estimate cluster structure and thus perform reasonably well over classical k-means. Thus, extensive experiments on synthetic datasets and real datasets demonstrate that GPD k-means outperforms competitors. The github codes are released in https://github.com/sixiaozheng/EVT-K-means.
School of Automation Science and Engineering, Xi’an Jiaotong University(西安交通大学自动化科学与工程学院);China Mobile Xiong’an Information and Communication Technology Co., Ltd.(中国移动雄安信息通信技术有限公司);China Mobile Information System Integration Co., Ltd.(中国移动信息系统集成有限公司)
Autonomous Cross Domain Adaptation under Extreme Label Scarcity
Nanyang Technological University(南洋理工大学);University of South Australia(南澳大学);Monash University(莫纳什大学);Singapore Institute of Manufacturing Technology, A*Star(新加坡科技研究局新加坡制造技术研究院)
Robust Ensemble Clustering Using Probability Trajectories
Sun Yat-sen University(中山大学);South China Agricultural University(华南农业大学);Guangdong Key Laboratory of Information Security Technology(广东省信息安全技术重点实验室);SYSU-CMU Shunde International Joint Research Institute (JRI)(中山大学-卡内基梅隆大学顺德国际联合研究院(JRI))
Multi-label Classification via Adaptive Resonance Theory-based Clustering
Graduate School of Engineering, Osaka Prefecture University(大阪府立大学工学院);Faculty of Computer Science and Information Technology, University of Malaya(马来亚大学计算机科学与信息技术学院);Southern University of Science and Technology(南方科技大学)
Bagged $k$-Distance for Mode-Based Clustering Using the Probability of Localized Level Sets
University of Twente(特文特大学)
在本文中,我们提出了一个集成学习算法命名为\textit{bagged $k$-distance for mode-based clustering}(\textit{BDMBC})通过提出一个新的测量称为\textit{probability of localized level sets}(\textit{PLLS}),这使我们能够找到所有的集群为不同的密度与一个全局阈值。在理论方面,我们表明,与适当选择的最近邻数$k_D$在袋装$k$ -距离,子样本大小$s$,装袋轮$B$,和最近邻数$k_L$的本地化水平集,BDMBC可以实现最佳的模式估计的收敛速度。事实证明,在相对小的$B$的情况下,子样本大小$s$可以远小于每个装袋轮的训练数据$n$的数量,并且可以同时减少最近邻居$k_D$的数量。此外,我们建立的水平集估计的PLLS的Hausdorff距离方面的最佳收敛结果,这表明,BDMBC可以找到本地化的水平集的不同密度,从而享有局部自适应性。在实际应用方面,通过数值实验验证了BDMBC算法在模式估计和水平集估计中的有效性,证明了算法的准确性和有效性.
Understanding InfoNCE: Transition Probability Matrix Induced Feature Clustering
organization= School of Mathematics;Computational Science, Xiangtan University , addressline= Yuhu District , city= Xiangtan , postcode= 411105 , state= Hunan Province , country= China;organization= School of Computer Science \& School of Cyberspace Science, Xiangtan University , addressline= Yuhu District , city= Xiangtan , postcode= 411105 , state= Hunan Province , country= China;organization= Investigation Department, Hunan Police Academy , addressline= Xingsha District , city= Changsha , postcode= 410138 , state= Hunan Province , country= China
Instituto Federal de Educação, Ciência e Tecnologia do Ceará(塞阿拉联邦教育与科学技术学院);Federal University of Ceara(塞阿拉联邦大学);Sigma Nova Science(Sigma Nova Science公司)
Automatic topography of high-dimensional data sets by non-parametric Density Peak clustering
SISSA, Scuola Internazionale Superiore Studi Avanzati(国际高级研究学院(SISSA));Functional Genomics Center Zurich(苏黎世功能基因组学中心);UZH/ETHZ(苏黎世大学/苏黎世联邦理工学院);Swiss Institute of Bioinformatics(瑞士生物信息学研究所);ICTP, International Centre for Theoretical Physics(国际理论物理中心(ICTP))
Aix Marseille Univ, CNRS, Centrale Marseille, I2M, Marseille, France(艾克斯马赛大学、法国国家科学研究中心、马赛中央理工学院、马赛数学研究所、法国马赛);National Research University Higher School of Economics(俄罗斯高等经济大学国立研究型大学)