We propose Few-Example Clustering (FEC), a novel algorithm that performs contrastive learning to cluster few examples. Our method is composed of the following three steps: (1) generation of candidate cluster assignments, (2) contrastive learning for each cluster assignment, and (3) selection of the best candidate. Based on the hypothesis that the contrastive learner with the ground-truth cluster assignment is trained faster than the others, we choose the candidate with the smallest training loss in the early stage of learning in step (3). Extensive experiments on the \textit{mini}-ImageNet and CUB-200-2011 datasets show that FEC outperforms other baselines by about 3.2% on average under various scenarios. FEC also exhibits an interesting learning curve where clustering performance gradually increases and then sharply drops.
Contrastive learning has recently attracted plenty of attention in deep graph clustering for its promising performance. However, complicated data augmentations and time-consuming graph convolutional operation undermine the efficiency of these methods. To solve this problem, we propose a Simple Contrastive Graph Clustering (SCGC) algorithm to improve the existing methods from the perspectives of network architecture, data augmentation, and objective function. As to the architecture, our network includes two main parts, i.e., pre-processing and network backbone. A simple low-pass denoising opera…
Deep Contrastive Graph Learning with Clustering-Oriented Guidance
Northwestern Polytechnical University（西北工业大学）；Key Laboratory of Intelligent Interaction and Applications, Ministry of Industry and Information Technology（工业和信息化部智能交互与应用重点实验室）
South China Agricultural University（华南农业大学）；Sun Yat-sen University（中山大学）
Recently the deep learning has shown its advantage in representation learning and clustering for time series data. Despite the considerable progress, the existing deep time series clustering approaches mostly seek to train the deep neural network by some instance reconstruction based or cluster distribution based objective, which, however, lack the ability to exploit the sample-wise (or augmentation-wise) contrastive information or even the higher-level (e.g., cluster-level) contrastiveness for learning discriminative and clustering-friendly representations. In light of this, this paper presen…
Most dialogue systems in real world rely on predefined intents and answers for QA service, so discovering potential intents from large corpus previously is really important for building such dialogue services. Considering that most scenarios have few intents known already and most intents waiting to be discovered, we focus on semi-supervised text clustering and try to make the proposed method benefit from labeled samples for better overall clustering performance. In this paper, we propose Deep Contrastive Semi-supervised Clustering (DCSC), which aims to cluster text samples in a semi-supervise…
University of Electronic Science and Technology of China（电子科技大学）
With the explosive growth of information technology, multi-view graph data have become increasingly prevalent and valuable. Most existing multi-view clustering techniques either focus on the scenario of multiple graphs or multi-view attributes. In this paper, we propose a generic framework to cluster multi-view attributed graph data. Specifically, inspired by the success of contrastive learning, we propose multi-view contrastive graph clustering (MCGC) method to learn a consensus graph since the original graph could be noisy or incomplete and is not directly applicable. Our method composes of …
We present federated momentum contrastive clustering (FedMCC), a learning framework that can not only extract discriminative representations over distributed local data but also perform data clustering. In FedMCC, a transformed data pair passes through both the online and target networks, resulting in four representations over which the losses are determined. The resulting high-quality representations generated by FedMCC can outperform several existing self-supervised learning methods for linear evaluation and semi-supervised learning tasks. FedMCC can easily be adapted to ordinary centralized…
South China Agricultural University（华南农业大学）；Sun Yat-sen University（中山大学）
Vision Transformer (ViT) has shown its advantages over the convolutional neural network (CNN) with its ability to capture global long-range dependencies for visual representation learning. Besides ViT, contrastive learning is another popular research topic recently. While previous contrastive learning works are mostly based on CNNs, some recent studies have attempted to combine ViT and contrastive learning for enhanced self-supervised learning. Despite the considerable progress, these combinations of ViT and contrastive learning mostly focus on the instance-level contrastiveness, which often o…
Shandong University（山东大学）；Joint SDU-NTU Centre for Artificial Intelligence Research (C-FAIR), Shandong University（山东大学-南洋理工大学联合人工智能研究中心（C-FAIR））；Lund University（隆德大学）；Nanyang Technological University（南洋理工大学）
Learning semantic-rich representations from raw unlabeled time series data is critical for downstream tasks such as classification and forecasting. Contrastive learning has recently shown its promising representation learning capability in the absence of expert annotations. However, existing contrastive approaches generally treat each instance independently, which leads to false negative pairs that share the same semantics. To tackle this problem, we propose MHCCL, a Masked Hierarchical Cluster-wise Contrastive Learning model, which exploits semantic information obtained from the hierarchical …
Subspace Clustering on Incomplete Data with Self-Supervised Contrastive Learning
Department of Electrical Engineering, Biostatistics（电气工程与生物统计学系）；Wisconsin Institute of Discovery（威斯康星发现研究所）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）
Contrastive learning is among the most successful methods for visual representation learning, and its performance can be further improved by jointly performing clustering on the learned representations. However, existing methods for joint clustering and contrastive learning do not perform well on long-tailed data distributions, as majority classes overwhelm and distort the loss of minority classes, thus preventing meaningful representations to be learned. Motivated by this, we develop a novel joint clustering and contrastive learning framework by adapting the debiased contrastive loss to avoid…
South China Agricultural University（华南农业大学）；Sun Yat-sen University（中山大学）
Deep clustering has attracted increasing attention in recent years due to its capability of joint representation learning and clustering via deep neural networks. In its latest developments, the contrastive learning has emerged as an effective technique to substantially enhance the deep clustering performance. However, the existing contrastive learning based deep clustering algorithms mostly focus on some carefully-designed augmentations (often with limited transformations to preserve the structure), referred to as weak augmentations, but cannot go beyond the weak augmentations to explore the …
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Massachusetts Institute of Technology（麻省理工学院）；Inria Paris（法国国家信息与自动化研究所巴黎中心）；Daimler TSS（戴姆勒TSS公司）
A good clustering algorithm can discover natural groupings in data. These groupings, if used wisely, provide a form of weak supervision for learning representations. In this work, we present Clustering-based Contrastive Learning (CCL), a new clustering-based representation learning approach that uses labels obtained from clustering along with video constraints to learn discriminative face features. We demonstrate our method on the challenging task of learning representations for video face clustering. Through several ablation studies, we analyze the impact of creating pair-wise positive and ne…
This paper proposes to perform online clustering by conducting twin contrastive learning (TCL) at the instance and cluster level. Specifically, we find that when the data is projected into a feature space with a dimensionality of the target cluster number, the rows and columns of its feature matrix correspond to the instance and cluster representation, respectively. Based on the observation, for a given dataset, the proposed TCL first constructs positive and negative pairs through data augmentations. Thereafter, in the row and column space of the feature matrix, instance- and cluster-level con…
Unsupervised clustering aims at discovering the semantic categories of data according to some distance measured in the representation space. However, different categories often overlap with each other in the representation space at the beginning of the learning process, which poses a significant challenge for distance-based clustering in achieving good separation between different categories. To this end, we propose Supporting Clustering with Contrastive Learning (SCCL) -- a novel framework to leverage contrastive learning to promote better separation. We assess the performance of SCCL on shor…
Contrastive Continual Multi-view Clustering with Filtered Structural Fusion
School of Computer, National University of Defense Technology（国防科技大学计算机学院）；School of Computer Science and Technology, Harbin University of Science and Technology（哈尔滨理工大学计算机科学与技术学院）
Hybrid-Collaborative Augmentation and Contrastive Sample Adaptive-Differential Awareness for Robust Attributed Graph Clustering
College of Information Science and Technology, Beijing University of Chemical Technology（北京化工大学信息科学与技术学院）；Beijing University of Technology（北京工业大学）；The University of Sydney Business School, The University of Sydney（悉尼大学商学院）
North China Electric Power University（华北电力大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；Beihang University（北京航空航天大学）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）
eMargin: Revisiting Contrastive Learning with Margin-Based Separation
Department of Computer Science, Norwegian University of Science and Technology, Norway（挪威科学技术大学计算机科学系）；Department of Computer Science, United States Naval Academy, USA（美国海军学院计算机科学系）
Sichuan University（四川大学）；TAL Education Group（好未来教育集团）；Institute for Infocomm Research, A*STAR（新加坡科技研究局资讯通信研究院）
In this paper, we propose a one-stage online clustering method called Contrastive Clustering (CC) which explicitly performs the instance- and cluster-level contrastive learning. To be specific, for a given dataset, the positive and negative instance pairs are constructed through data augmentations and then projected into a feature space. Therein, the instance- and cluster-level contrastive learning are respectively conducted in the row and column space by maximizing the similarities of positive pairs while minimizing those of negative ones. Our key observation is that the rows of the feature m…
State Key Laboratory for Novel Software Technology, Nanjing University（南京大学现代软件技术国家重点实验室）；School of Information Technology and Electrical Engineering, The University of Queensland（昆士兰大学信息技术与电气工程学院）
Incomplete multi-view clustering (IMVC) is an unsupervised approach, among which IMVC via contrastive learning has received attention due to its excellent performance. The previous methods have the following problems: 1) Over-reliance on additional projection heads when solving the dimensional collapse problem in which latent features are only valid in lower-dimensional subspaces during clustering. However, many parameters in the projection heads are unnecessary. 2) The recovered view contain inconsistent private information and useless private information will mislead the learning of common s…
University of Electronic Science and Technology of China（电子科技大学）；Lehigh University（利哈伊大学）
Multi-view clustering can explore common semantics from multiple views and has attracted increasing attention. However, existing works punish multiple objectives in the same feature space, where they ignore the conflict between learning consistent common semantics and reconstructing inconsistent view-private information. In this paper, we propose a new framework of multi-level feature learning for contrastive multi-view clustering to address the aforementioned issue. Our method learns different levels of features from the raw features, including low-level features, high-level features, and sem…
China University of Geosciences（中国地质大学）；Helmholtz-Zentrum Dresden-Rossendorf（德累斯顿罗森多夫亥姆霍兹中心）；Helmholtz Institute Freiberg for Resource Technology（弗赖贝格资源技术亥姆霍兹研究所）；Institute of Advanced Research in Artificial Intelligence（人工智能高级研究院）；National University of Defense Technology（国防科技大学）
Clustering of hyperspectral images is a fundamental but challenging task. The recent development of hyperspectral image clustering has evolved from shallow models to deep and achieved promising results in many benchmark datasets. However, their poor scalability, robustness, and generalization ability, mainly resulting from their offline clustering scenarios, greatly limit their application to large-scale hyperspectral data. To circumvent these problems, we present a scalable deep online clustering model, named Spectral-Spatial Contrastive Clustering (SSCC), based on self-supervised learning. S…
scAGC: Learning Adaptive Cell Graphs with Contrastive Guidance for Single-Cell Clustering
Shanghai Key Laboratory of Trustworthy Computing, East China Normal University, Shanghai, China（上海可信计算实验室，华东师范大学）；Department of Computational Biology, Mohamed bin Zayed University of AI, Abu Dhabi, UAE（计算生物学系，马尔代夫人工智能大学）；Department of Computer Science, Stevens Institute of Technology, Hoboken, USA（计算机科学系，史蒂文斯理工学院）；College of Electronics and Information Engineering, Tongji University, Shanghai, China（电子信息工程学院，同济大学）
National University of Defense Technology（国防科技大学）；University of Science and Technology of China（中国科学技术大学）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）
Benefiting from the intrinsic supervision information exploitation capability, contrastive learning has achieved promising performance in the field of deep graph clustering recently. However, we observe that two drawbacks of the positive and negative sample construction mechanisms limit the performance of existing algorithms from further improvement. 1) The quality of positive samples heavily depends on the carefully designed data augmentations, while inappropriate data augmentations would easily lead to the semantic drift and indiscriminative positive samples. 2) The constructed negative samp…
Contrastive learning is a promising approach to unsupervised learning, as it inherits the advantages of well-studied deep models without a dedicated and complex model design. In this paper, based on bidirectional encoder representations from transformers, we propose self-supervised contrastive learning (SCL) as well as few-shot contrastive learning (FCL) with unsupervised data augmentation (UDA) for text clustering. SCL outperforms state-of-the-art unsupervised clustering approaches for short texts and those for long texts in terms of several clustering evaluation measures. FCL achieves perfor…
Unsupervised disentangled representation learning is a long-standing problem in computer vision. This work proposes a novel framework for performing image clustering from deep embeddings by combining instance-level contrastive learning with a deep embedding based cluster center predictor. Our approach jointly learns representations and predicts cluster centers in an end-to-end manner. This is accomplished via a three-pronged approach that combines a clustering loss, an instance-wise contrastive loss, and an anchor loss. Our fundamental intuition is that using an ensemble loss that incorporates…
SMART: Semantic Matching Contrastive Learning for Partially View-Aligned Clustering
College of Computer Science and Technology, Huaqiao University（华侨大学计算机科学与技术学院）；Department of Computer Science, Shantou University（汕头大学计算机科学系）；Chongqing Key Laboratory of Nonlinear Circuits and Intelligent Information Processing, College of Electronic and Information Engineering, Southwest University（西南大学电子信息工程学院非线性电路与智能信息处理重点实验室）；School of Computer Science and Engineering, South China University of Technology（华南理工大学计算机科学与工程学院）；Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）
Mask-informed Deep Contrastive Incomplete Multi-view Clustering
Faculty of Data Science, City University of Macau（澳门城市大学数据科学学院）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；School of Computer Science, China University of Geosciences（中国地质大学计算机学院）
