通知：联邦节假日调休：哥伦布日 / 原住民日（Columbus Day / Indigenous Peoples' Day），今日（13日）无更新。通知：联邦节假日调休：哥伦布日 / 原住民日（Columbus Day / Indigenous Peoples' Day），今日（13日）无更新。
arXiv 2512.07880cs.LG
基于正交原型的半监督对比学习
Semi-Supervised Contrastive Learning with Orthonormal Prototypes
Department of Electrical Engineering, Statistics, Biostatistics（电气工程系、统计学与生物统计学系）；Wisconsin Institute of Discovery（威斯康星发现研究所）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）
Contrastive learning has emerged as a powerful method in deep learning, excelling at learning effective representations through contrasting samples from different distributions. However, dimensional collapse, where embeddings converge into a lower-dimensional space, poses a significant challenge, especially in semi-supervised and self-supervised setups. In this paper, we first identify a critical learning-rate threshold, beyond which standard contrastive losses converge to collapsed solutions. Building on these insights, we propose CLOP, a novel semi-supervised loss function designed to prevent dimensional collapse by promoting the formation of orthogonal linear subspaces among class embeddings. Through extensive experiments on real and synthetic datasets, we demonstrate that CLOP improves performance in image classification and object detection tasks while also exhibiting greater stability across different learning rates and batch sizes.
Breaking the Prototype Bias Loop: Confidence-Aware Federated Contrastive Learning for Highly Imbalanced Clients
Key Laboratory of Water Big Data Technology of Ministry of Water Resources, College of Computer Science and Software Engineering, Hohai University, Nanjing, China（水利部水大数据技术重点实验室，计算机科学与软件工程学院，河海大学，南京，中国）；Department of Computer Science, City University of Hong Kong, Hong Kong, China（计算机科学系，香港城市大学，香港，中国）；State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China（新型软件技术国家重点实验室，南京大学，南京，中国）；School of Artificial Intelligence and Information Technology, Nanjing University of Chinese Medicine, Nanjing, China（人工智能与信息技术学院，南京中医药大学，南京，中国）
Graph Neural Networks (GNNs) have been widely used to learn node representations and with outstanding performance on various tasks such as node classification. However, noise, which inevitably exists in real-world graph data, would considerably degrade the performance of GNNs as the noise is easily propagated via the graph structure. In this work, we propose a novel and robust method, Bayesian Robust Graph Contrastive Learning (BRGCL), which trains a GNN encoder to learn robust node representations. The BRGCL encoder is a completely unsupervised encoder. Two steps are iteratively executed at e…
This paper presents Prototypical Contrastive Learning (PCL), an unsupervised representation learning method that addresses the fundamental limitations of instance-wise contrastive learning. PCL not only learns low-level features for the task of instance discrimination, but more importantly, it implicitly encodes semantic structures of the data into the learned embedding space. Specifically, we introduce prototypes as latent variables to help find the maximum-likelihood estimation of the network parameters in an Expectation-Maximization framework. We iteratively perform E-step as finding the di…
BrainSCL: Subtype-Guided Contrastive Learning for Brain Disorder Diagnosis
Computer Science and Engineering, Northeastern University, Shenyang, China（东北大学计算机科学与工程学院）；School of Computer Science and Artificial Intelligence, Shandong Normal University, Jinan, China（山东师范大学计算机科学与人工智能学院）；AiShiWeiLai AI Research, Beijing, China（艾世维来人工智能研究所）；Early Intervention Unit, Department of Psychiatry, Affiliated Nanjing Brain Hospital, Nanjing Medical University, Nanjing, China（南京医科大学附属南京脑科医院早期干预科）；Amii, University of Alberta, Edmonton, AB, Canada（阿尔伯塔大学Amii）
FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learning
Shanghai Jiao Tong University（上海交通大学）；Institute for AI Industry Research, Tsinghua University（清华大学智能产业研究院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Queen’s University Belfast（英国女王大学）
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；ETH Zürich（苏黎世联邦理工学院）；The Chinese University of Hong Kong（香港中文大学）
Standard few-shot relation classification (RC) is designed to learn a robust classifier with only few labeled data for each class. However, previous works rarely investigate the effects of a different number of classes (i.e., $N$-way) and number of labeled data per class (i.e., $K$-shot) during training vs. testing. In this work, we define a new task, \textit{inconsistent few-shot RC}, where the model needs to handle the inconsistency of $N$ and $K$ between training and testing. To address this new task, we propose Prototype Network-based cross-attention contrastive learning (ProtoCACL) to cap…
Australian Artificial Intelligence Institute（澳大利亚人工智能研究所）；University of Technology Sydney（悉尼科技大学）；Google Research（谷歌研究院）；University of Washington（华盛顿大学）；University of Maryland（马里兰大学）
Federated Learning (FL) is a machine learning paradigm that allows decentralized clients to learn collaboratively without sharing their private data. However, excessive computation and communication demands pose challenges to current FL frameworks, especially when training large-scale models. To prevent these issues from hindering the deployment of FL systems, we propose a lightweight framework where clients jointly learn to fuse the representations generated by multiple fixed pre-trained models rather than training a large-scale model from scratch. This leads us to a more practical FL problem…
Contrastive self-supervised learning (CSL) with a prototypical regularization has been introduced in learning meaningful representations for downstream tasks that require strong semantic information. However, to optimize CSL with a loss that performs the prototypical regularization aggressively, e.g., the ProtoNCE loss, might cause the "coagulation" of examples in the embedding space. That is, the intra-prototype diversity of samples collapses to trivial solutions for their prototype being well-separated from others. Motivated by previous works, we propose to mitigate this phenomenon by learni…
The success of existing multi-view clustering relies on the assumption of sample integrity across multiple views. However, in real-world scenarios, samples of multi-view are partially available due to data corruption or sensor failure, which leads to incomplete multi-view clustering study (IMVC). Although several attempts have been proposed to address IMVC, they suffer from the following drawbacks: i) Existing methods mainly adopt cross-view contrastive learning forcing the representations of each sample across views to be exactly the same, which might ignore view discrepancy and flexibility i…
Sun Yat-sen University（中山大学）；Sea AI Lab（Sea人工智能实验室）；Tencent（腾讯）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Graph-level representations are critical in various real-world applications, such as predicting the properties of molecules. But in practice, precise graph annotations are generally very expensive and time-consuming. To address this issue, graph contrastive learning constructs instance discrimination task which pulls together positive pairs (augmentation pairs of the same graph) and pushes away negative pairs (augmentation pairs of different graphs) for unsupervised representation learning. However, since for a query, its negatives are uniformly sampled from all graphs, existing methods suffer…
Contrastive Self-supervised Learning (CSL) is a practical solution that learns meaningful visual representations from massive data in an unsupervised approach. The ordinary CSL embeds the features extracted from neural networks onto specific topological structures. During the training progress, the contrastive loss draws the different views of the same input together while pushing the embeddings from different inputs apart. One of the drawbacks of CSL is that the loss term requires a large number of negative samples to provide better mutual information bound ideally. However, increasing the nu…
Contrastive learning has gained popularity as an effective self-supervised representation learning technique. Several research directions improve traditional contrastive approaches, e.g., prototypical contrastive methods better capture the semantic similarity among instances and reduce the computational burden by considering cluster prototypes or cluster assignments, while adversarial instance-wise contrastive methods improve robustness against a variety of attacks. To the best of our knowledge, no prior work jointly considers robustness, cluster-wise semantic similarity and computational effi…
SPORT: Structure-Aware Prototype Disentanglement for Incomplete Multi-View Clustering
College of Computer and Cyberspace Security, Hebei Normal University（河北师范大学计算机与网络空间安全学院）；Department of Statistics and Data Science, Southern University of Science and Technology（南方科技大学统计与数据科学系）；Key Laboratory of Tibetan Information Processing, Ministry of Education, Qinghai Normal University（青海师范大学藏文信息处理教育部重点实验室）；School of Computer Science and Technology, Beijing Jiaotong University（北京交通大学计算机科学与技术学院）；State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所多模态人工智能系统国家重点实验室）
PAST-TIDE: Prototype-Anchored Statement Tuning with Topic-Invariant Normalization for Stance Detection
Department of Computer Science and Engineering, Khulna University of Engineering & Technology（库尔纳工程技术大学计算机科学与工程系）；Department of Electrical and Computer Engineering, University of Waterloo（滑铁卢大学电气与计算机工程系）；Department of Machine Learning, Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学机器学习系）
Zhejiang University（浙江大学）；Chongqing University（重庆大学）；RIKEN Center for Advanced Intelligence Project（理化学研究所先进智能项目中心）
Partial label learning (PLL) is an important problem that allows each training example to be labeled with a coarse candidate set, which well suits many real-world data annotation scenarios with label ambiguity. Despite the promise, the performance of PLL often lags behind the supervised counterpart. In this work, we bridge the gap by addressing two key research challenges in PLL -- representation learning and label disambiguation -- in one coherent framework. Specifically, our proposed framework PiCO consists of a contrastive learning module along with a novel class prototype-based label disam…
The pretasks are mainly built on mutual information estimation, which requires data augmentation to construct positive samples with similar semantics to learn invariant signals and negative samples with dissimilar semantics in order to empower representation discriminability. However, an appropriate data augmentation configuration depends heavily on lots of empirical trials such as choosing the compositions of data augmentation techniques and the corresponding hyperparameter settings. We propose an augmentation-free graph contrastive learning method, invariant-discriminative graph contrastive …
Massive rumors usually appear along with breaking news or trending topics, seriously hindering the truth. Existing rumor detection methods are mostly focused on the same domain, and thus have poor performance in cross-domain scenarios due to domain shift. In this work, we propose an end-to-end instance-wise and prototype-wise contrastive learning model with a cross-attention mechanism for cross-domain rumor detection. The model not only performs cross-domain feature alignment but also enforces target samples to align with the corresponding prototypes of a given source domain. Since target labe…
In this study, we investigate self-supervised representation learning for speaker verification (SV). First, we examine a simple contrastive learning approach (SimCLR) with a momentum contrastive (MoCo) learning framework, where the MoCo speaker embedding system utilizes a queue to maintain a large set of negative examples. We show that better speaker embeddings can be learned by momentum contrastive learning. Next, alternative augmentation strategies are explored to normalize extrinsic speaker variabilities of two random segments from the same speech utterance. Specifically, augmentation in th…
Many clinical deep learning algorithms are population-based and difficult to interpret. Such properties limit their clinical utility as population-based findings may not generalize to individual patients and physicians are reluctant to incorporate opaque models into their clinical workflow. To overcome these obstacles, we propose to learn patient-specific embeddings, entitled patient cardiac prototypes (PCPs), that efficiently summarize the cardiac state of the patient. To do so, we attract representations of multiple cardiac signals from the same patient to the corresponding PCP via supervise…
Intelligently Augmented Contrastive Tensor Factorization: Empowering Multi-dimensional Time Series Classification in Low-Data Environments
Engineering Product Development Singapore University of Technology and Design（新加坡科技与设计大学工程产品开发学院）；School of Automation Chongqing University（重庆大学自动化学院）；Institute for Infocomm Research Agency for Science, Technology and Research（信息与通信研究所）；School of Electrical and Electronics Engineering Nanyang Technological University（南洋理工大学电子与电气工程学院）
Semi-Supervised Federated Learning via Dual Contrastive Learning and Soft Labeling for Intelligent Fault Diagnosis
School of Electronic and Optical Engineering, Nanjing University of Science and Technology（南京理工大学电子与光学工程学院）；School of Information Science and Engineering, Southeast University（东南大学信息科学与工程学院）；Institute of Artificial Intelligence Research, Jiangsu Second Normal University（江苏第二师范学院人工智能研究所）；Digital Research Institute, ENN Group（ENN集团数字研究院）；School of Computing, Engineering and Mathematical Sciences, La Trobe University（拉筹伯大学计算、工程与数学科学学院）；College of Science and Engineering, James Cook University（詹姆斯库克大学科学与工程学院）
Boundary-aware Prototype-driven Adversarial Alignment for Cross-Corpus EEG Emotion Recognition
School of Biological Science and Medical Engineering, Hunan University of Technology（湖南工业大学生物科学与医学工程学院）；School of Biomedical Engineering, Health Science Center, Shenzhen University（深圳大学医学部生物医学工程学院）；Guangdong Provincial Key Laboratory of Biomedical Measurements and Ultrasound Imaging（广东省生物医学测量与超声成像重点实验室）；Shenzhen Pengrui Brain Science Technology（深圳鹏瑞脑科学技术有限公司）
Shandong University（山东大学）；Joint SDU-NTU Centre for Artificial Intelligence Research (C-FAIR), Shandong University（山东大学-南洋理工大学联合人工智能研究中心（C-FAIR））；Lund University（隆德大学）；Nanyang Technological University（南洋理工大学）
Learning semantic-rich representations from raw unlabeled time series data is critical for downstream tasks such as classification and forecasting. Contrastive learning has recently shown its promising representation learning capability in the absence of expert annotations. However, existing contrastive approaches generally treat each instance independently, which leads to false negative pairs that share the same semantics. To tackle this problem, we propose MHCCL, a Masked Hierarchical Cluster-wise Contrastive Learning model, which exploits semantic information obtained from the hierarchical …
PRCL: Probabilistic Representation Contrastive Learning for Semi-Supervised Semantic Segmentation
School of Computer Science and Engineering, Northeastern University（东北大学计算机科学与工程学院）；Intelligent Game and Decision Laboratory（智能游戏与决策实验室）；Alibaba DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）；Zhejiang University（浙江大学）
Medical image annotation typically requires expert knowledge and hence incurs time-consuming and expensive data annotation costs. To alleviate this burden, we propose a novel learning scenario, Exemplar Learning (EL), to explore automated learning processes for medical image segmentation with a single annotated image example. This innovative learning task is particularly suitable for medical image segmentation, where all categories of organs can be presented in one single image and annotated all at once. To address this challenging EL task, we propose an Exemplar Learning-based Synthesis Net (…
Contrastive Continual Learning with Importance Sampling and Prototype-Instance Relation Distillation
Sun Yat-sen University（中山大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）
Transformations in the input space of Deep Neural Networks (DNN) lead to unintended changes in the feature space. Almost perceptually identical inputs, such as adversarial examples, can have significantly distant feature representations. On the contrary, Out-of-Distribution (OOD) samples can have highly similar feature representations to training set samples. Our theoretical analysis for DNNs trained with a categorical classification head suggests that the inflexible logit space restricted by the classification problem size is one of the root causes for the lack of $\textit{robustness}$. Our s…
MVCL-DAF++: Enhancing Multimodal Intent Recognition via Prototype-Aware Contrastive Alignment and Coarse-to-Fine Dynamic Attention Fusion
University of Shanghai for Science and TechnologyChina（上海科学技术大学中国）；Shenzhen Institute of Advanced Technology, Chinese Academy of SciencesChina（深圳先进技术研究院，中国科学院中国）；University of Minnesota-Twin Cities, USA（明尼苏达大学双城分校，美国）；University of LeedsUK（利兹大学，英国）
THESAURUS: Contrastive Graph Clustering by Swapping Fused Gromov-Wasserstein Couplings
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；School of Systems Science and Engineering, Sun Yat-sen University（中山大学系统科学与工程学院）
Multi-Normal Prototypes Learning for Weakly Supervised Anomaly Detection
School of Software and Microelectronics Peking University（北京大学软件与微电子学院）；School of Computer Science Peking University（北京大学计算机学院）；National Engineering Center of Software Engineering Peking University（北京大学软件工程国家工程研究中心）
Data Science Group, IBS（基础科学研究院数据科学组）；School of Computing, KAIST（韩国科学技术院计算学院）
Anomaly detection aims at identifying deviant instances from the normal data distribution. Many advances have been made in the field, including the innovative use of unsupervised contrastive learning. However, existing methods generally assume clean training data and are limited when the data contain unknown anomalies. This paper presents Elsa, a novel semi-supervised anomaly detection approach that unifies the concept of energy-based models with unsupervised contrastive learning. Elsa instills robustness against any data contamination by a carefully designed fine-tuning step based on the new …
