We introduce supervised contrastive active learning (SCAL) and propose efficient query strategies in active learning based on the feature similarity (featuresim) and principal component analysis based feature-reconstruction error (fre) to select informative data samples with diverse feature representations. We demonstrate our proposed method achieves state-of-the-art accuracy, model calibration and reduces sampling bias in an active learning setup for balanced and imbalanced datasets on image classification tasks. We also evaluate robustness of model to distributional shift derived from different query strategies in active learning setting. Using extensive experiments, we show that our proposed approach outperforms high performing compute-intensive methods by a big margin resulting in 9.9% lower mean corruption error, 7.2% lower expected calibration error under dataset shift and 8.9% higher AUROC for out-of-distribution detection.
Samsung Semiconductor US（美国三星半导体）；National Yang Ming Chiao Tung University（国立阳明交通大学）
Point cloud data plays an essential role in robotics and self-driving applications. Yet, annotating point cloud data is time-consuming and nontrivial while they enable learning discriminative 3D representations that empower downstream tasks, such as classification and segmentation. Recently, contrastive learning-based frameworks have shown promising results for learning 3D representations in a self-supervised manner. However, existing contrastive learning methods cannot precisely encode and associate structural features and search the higher dimensional augmentation space efficiently. In this …
Sun Yat-sen University（中山大学）；Sea AI Lab（Sea人工智能实验室）；Tencent（腾讯）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Graph-level representations are critical in various real-world applications, such as predicting the properties of molecules. But in practice, precise graph annotations are generally very expensive and time-consuming. To address this issue, graph contrastive learning constructs instance discrimination task which pulls together positive pairs (augmentation pairs of the same graph) and pushes away negative pairs (augmentation pairs of different graphs) for unsupervised representation learning. However, since for a query, its negatives are uniformly sampled from all graphs, existing methods suffer…
RouterDC: Query-Based Router by Dual Contrastive Learning for Assembling Large Language Models
Southern University of Science and Technology（南方科技大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）
For an image query, unsupervised contrastive learning labels crops of the same image as positives, and other image crops as negatives. Although intuitive, such a native label assignment strategy cannot reveal the underlying semantic similarity between a query and its positives and negatives, and impairs performance, since some negatives are semantically similar to the query or even share the same semantic class as the query. In this work, we first prove that for contrastive learning, inaccurate label assignment heavily impairs its generalization for semantic instance discrimination, while accu…
Query-based video grounding is an important yet challenging task in video understanding, which aims to localize the target segment in an untrimmed video according to a sentence query. Most previous works achieve significant progress by addressing this task in a fully-supervised manner with segment-level labels, which require high labeling cost. Although some recent efforts develop weakly-supervised methods that only need the video-level knowledge, they generally match multiple pre-defined segment proposals with query and select the best one, which lacks fine-grained frame-level details for dis…
Decomposing Query-Key Feature Interactions Using Contrastive Covariances
Harvard University（哈佛大学）；Technion - Israel Institute of Technology（技术ion-以色列理工学院）；Kempner Institute, Harvard University（凯普纳研究所，哈佛大学）；Google DeepMind（谷歌DeepMind）
The Chinese University of Hong Kong（香港中文大学）；Tencent AI Lab（腾讯AI实验室）
Exemplar-Guided Paraphrase Generation (EGPG) aims to generate a target sentence which conforms to the style of the given exemplar while encapsulating the content information of the source sentence. In this paper, we propose a new method with the goal of learning a better representation of the style andthe content. This method is mainly motivated by the recent success of contrastive learning which has demonstrated its power in unsupervised feature extraction tasks. The idea is to design two contrastive losses with respect to the content and the style by considering two problem characteristics d…
BrainSCL: Subtype-Guided Contrastive Learning for Brain Disorder Diagnosis
Computer Science and Engineering, Northeastern University, Shenyang, China（东北大学计算机科学与工程学院）；School of Computer Science and Artificial Intelligence, Shandong Normal University, Jinan, China（山东师范大学计算机科学与人工智能学院）；AiShiWeiLai AI Research, Beijing, China（艾世维来人工智能研究所）；Early Intervention Unit, Department of Psychiatry, Affiliated Nanjing Brain Hospital, Nanjing Medical University, Nanjing, China（南京医科大学附属南京脑科医院早期干预科）；Amii, University of Alberta, Edmonton, AB, Canada（阿尔伯塔大学Amii）
Global-Graph Guided and Local-Graph Weighted Contrastive Learning for Unified Clustering on Incomplete and Noise Multi-View Data
Guangxi Normal University（广西师范学院）；University of Electronic Science and Technology of China（电子科技大学）；Singapore University of Technology and Design（新加坡科技设计大学）；Hainan University（海南大学）
PGCLODA: Prompt-Guided Graph Contrastive Learning for Oligopeptide-Infectious Disease Association Prediction
Institute of Physical Science and Information Technology（物理科学与信息技术研究所）；Key Laboratory of Intelligent Computing and Signal Processing of Ministry of Education（教育部智能计算与信号处理重点实验室）；School of Artificial Intelligence（人工智能学院）
Indian Institute of Information Technology, Sri City, Chittoor（印度信息技术学院，斯里城，奇图尔）；Indian Institute of Information Technology, Allahabad（印度信息技术学院，阿拉哈巴德）
Covid-19 global pandemic continues to devastate health care systems across the world. At present, the Covid-19 testing is costly and time-consuming. Chest X-Ray (CXR) testing can be a fast, scalable, and non-invasive method. The existing methods suffer due to the limited CXR samples available from Covid-19. Thus, inspired by the limitations of the open-source work in this field, we propose attention guided contrastive CNN architecture (AC-CovidNet) for Covid-19 detection in CXR images. The proposed method learns the robust and discriminative features with the help of contrastive loss. Moreover…
Context modeling plays a significant role in building multi-turn dialogue systems. In order to make full use of context information, systems can use Incomplete Utterance Rewriting(IUR) methods to simplify the multi-turn dialogue into single-turn by merging current utterance and context information into a self-contained utterance. However, previous approaches ignore the intent consistency between the original query and rewritten query. The detection of omitted or coreferred locations in the original query can be further improved. In this paper, we introduce contrastive learning and multi-task l…
National University of Defense Technology（国防科技大学）；University of Science and Technology of China（中国科学技术大学）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）
Benefiting from the intrinsic supervision information exploitation capability, contrastive learning has achieved promising performance in the field of deep graph clustering recently. However, we observe that two drawbacks of the positive and negative sample construction mechanisms limit the performance of existing algorithms from further improvement. 1) The quality of positive samples heavily depends on the carefully designed data augmentations, while inappropriate data augmentations would easily lead to the semantic drift and indiscriminative positive samples. 2) The constructed negative samp…
Faculty of Information Technology, University of Science, Ho Chi Minh City, Vietnam.（信息科技学院，科学大学，胡志明市，越南）；Vietnam National University, Ho Chi Minh City, Vietnam.（越南国家大学，胡志明市，越南）
LAMP: Learnable Meta-Path Guided Adversarial Contrastive Learning for Heterogeneous Graphs
University of New South Wales（新南威尔士大学）；Yonsei University（延世大学）；RIKEN AIP（理化学研究所人工智能项目）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Deep learning models are shown to be vulnerable to adversarial examples. Though adversarial training can enhance model robustness, typical approaches are computationally expensive. Recent works proposed to transfer the robustness to adversarial attacks across different tasks or models with soft labels.Compared to soft labels, feature contains rich semantic information and holds the potential to be applied to different downstream tasks. In this paper, we propose a novel approach called Guided Adversarial Contrastive Distillation (GACD), to effectively transfer adversarial robustness from teache…
State Key Laboratory for Novel Software Technology, Nanjing University（南京大学现代软件技术国家重点实验室）；School of Information Technology and Electrical Engineering, The University of Queensland（昆士兰大学信息技术与电气工程学院）
DFM: Difference Feature Modeling with Text-Guided Gated Contrastive Loss for Remote Sensing Image Change Captioning
ShanghaiTech University（上海科技大学）；National University of Defense Technology（国防科技大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Goldman Sachs（高盛）
The University of Melbourne（墨尔本大学）；National University of Singapore（新加坡国立大学）
Trajectory similarity measures act as query predicates in trajectory databases, making them the key player in determining the query results. They also have a heavy impact on the query efficiency. An ideal measure should have the capability to accurately evaluate the similarity between any two trajectories in a very short amount of time. Towards this aim, we propose a contrastive learning-based trajectory modeling method named TrajCL. We present four trajectory augmentation methods and a novel dual-feature self-attention-based trajectory backbone encoder. The resultant model can jointly learn b…
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；University Medicine Essen（埃森大学医学院）
When reading images, radiologists generate text reports describing the findings therein. Current state-of-the-art computer-aided diagnosis tools utilize a fixed set of predefined categories automatically extracted from these medical reports for training. This form of supervision limits the potential usage of models as they are unable to pick up on anomalies outside of their predefined set, thus, making it a necessity to retrain the classifier with additional data when faced with novel classes. In contrast, we investigate direct text supervision to break away from this closed set assumption. By…
Oracle-Guided Masked Contrastive Reinforcement Learning for Visuomotor Policies
School of Mechanical and Aerospace Engineering, Nanyang Technological University（南洋理工大学机械与航空航天工程学院）；College of Automation Engineering, Nanjing University of Aeronautics and Astronautics（南京航空航天大学自动化工程学院）
A Contrastive Learning-Guided Confident Meta-learning for Zero Shot Anomaly Detection
Dept. of Engineering for Innovation Medicine, University of Verona（创新医学工程系，威尼斯大学）；Faculty of Computer and Information Science, University of Ljubljana（计算机与信息科学系，卢布尔雅那大学）；Qualyco S.r.l.（Qualyco公司）
ETH Zurich（苏黎世联邦理工学院）；University of Hong Kong（香港大学）；HKUST（香港科技大学）；University of Trento（特伦托大学）；University of Wurzburg（维尔茨堡大学）
We propose a novel ECGAN for the challenging semantic image synthesis task. Although considerable improvement has been achieved, the quality of synthesized images is far from satisfactory due to three largely unresolved challenges. 1) The semantic labels do not provide detailed structural information, making it difficult to synthesize local details and structures. 2) The widely adopted CNN operations such as convolution, down-sampling, and normalization usually cause spatial resolution loss and thus cannot fully preserve the original semantic information, leading to semantically inconsistent r…
SaliencyI2PLoc: saliency-guided image-point cloud localization using contrastive learning
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing（测绘遥感信息工程国家重点实验室）；Wuhan University（武汉大学）；School of Electrical and Electronic Engineering（电气与电子工程学院）；Nanyang Technological University（南洋理工大学）；School of Geography and Environment（地理与环境学院）；Jiangxi Normal University（江西师范大学）
Facebook AI Research（脸书人工智能研究院）；Columbia University（哥伦比亚大学）；MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）；ITS, CUNY Graduate Center（纽约市立大学研究生中心智能交通系统实验室）
Self-supervised learning has recently begun to rival supervised learning on computer vision tasks. Many of the recent approaches have been based on contrastive instance discrimination (CID), in which the network is trained to recognize two augmented versions of the same instance (a query and positive) while discriminating against a pool of other instances (negatives). The learned representation is then used on downstream tasks such as image classification. Using methodology from MoCo v2 (Chen et al., 2020), we divided negatives by their difficulty for a given query and studied which difficulty…
