Homeomorphism Prior for False Positive and Negative Problem in Medical Image Dense Contrastive Representation Learning
Southeast University（东南大学）；Centre de Recherche en Information Biomédicale Sino-Français (CRIBs)（中法生物医学信息研究中心（CRIBs））；Jiangsu Provincial Joint International Research Laboratory of Medical Information Processing（江苏省医疗信息处理联合国际研究实验室）；School of Instrument Science and Engineering, Southeast University（东南大学仪器科学与工程学院）；Western University（西安大略大学）；Case Western Reserve University（凯斯西储大学）
Yuting He, Boyu Wang, Rongjun Ge, Yang Chen, Guanyu Yang, Shuo Li
Dense contrastive representation learning (DCRL) has greatly improved the learning efficiency for image-dense prediction tasks, showing its great potential to reduce the large costs of medical image collection and dense annotation. However, the properties of medical images make unreliable correspondence discovery, bringing an open problem of large-scale false positive and negative (FP&N) pairs in DCRL. In this paper, we propose GEoMetric vIsual deNse sImilarity (GEMINI) learning which embeds the homeomorphism prior to DCRL and enables a reliable correspondence discovery for effective dense contrast. We propose a deformable homeomorphism learning (DHL) which models the homeomorphism of medical images and learns to estimate a deformable mapping to predict the pixels' correspondence under topological preservation. It effectively reduces the searching space of pairing and drives an implicit and soft learning of negative pairs via a gradient. We also propose a geometric semantic similarity (GSS) which extracts semantic information in features to measure the alignment degree for the correspondence learning. It will promote the learning efficiency and performance of deformation, constructing positive pairs reliably. We implement two practical variants on two typical representation learning tasks in our experiments. Our promising results on seven datasets which outperform the existing methods show our great superiority. We will release our code on a companion link: https://github.com/YutingHe-list/GEMINI.
California Institute of Technology（加州理工学院）；University of Southern California（南加州大学）；Mississippi State University（密西西比州立大学）
Contrastive learning has recently demonstrated superior performance to supervised learning, despite requiring no training labels. We explore how contrastive learning can be applied to hundreds of thousands of unlabeled Mars terrain images, collected from the Mars rovers Curiosity and Perseverance, and from the Mars Reconnaissance Orbiter. Such methods are appealing since the vast majority of Mars images are unlabeled as manual annotation is labor intensive and requires extensive domain knowledge. Contrastive learning, however, assumes that any given pair of distinct images contain distinct sem…
Enhancing Contrastive Learning for Geolocalization by Discovering Hard Negatives on Semivariograms
Professorship Big Geospatial Data Management, Technical University of Munich Germany（技术大学慕尼黑大数据地理空间管理教授职位）；University of Maine United States（缅因大学）；University of the Bundeswehr Munich Germany（联邦国防军大学慕尼黑）
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Adobe（奥多比公司）；Carle Illinois College of Medicine（卡尔伊利诺伊医学院）
Existing vision-text contrastive learning like CLIP aims to match the paired image and caption embeddings while pushing others apart, which improves representation transferability and supports zero-shot prediction. However, medical image-text datasets are orders of magnitude below the general images and captions from the internet. Moreover, previous methods encounter many false negatives, i.e., images and reports from separate patients probably carry the same semantics but are wrongly treated as negatives. In this paper, we decouple images and texts for multimodal contrastive learning thus sca…
Learning Contrastive Feature Representations for Facial Action Unit Detection
National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China.（新型软件技术国家重点实验室，南京大学，中国南京）；School of Intelligence Science and Technology, Nanjing University, Suzhou, China.（智能科学与技术学院，南京大学，中国苏州）；School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu, China.（计算与人工智能学院，西南交通大学，中国成都）
Early detection of lung nodules with computed tomography (CT) is critical for the longer survival of lung cancer patients and better quality of life. Computer-aided detection/diagnosis (CAD) is proven valuable as a second or concurrent reader in this context. However, accurate detection of lung nodules remains a challenge for such CAD systems and even radiologists due to not only the variability in size, location, and appearance of lung nodules but also the complexity of lung structures. This leads to a high false-positive rate with CAD, compromising its clinical efficacy. Motivated by recent …
Contrastive learning allows us to flexibly define powerful losses by contrasting positive pairs from sets of negative samples. Recently, the principle has also been used to learn cross-modal embeddings for video and text, yet without exploiting its full potential. In particular, previous losses do not take the intra-modality similarities into account, which leads to inefficient embeddings, as the same content is mapped to multiple points in the embedding space. With CrossCLR, we present a contrastive loss that fixes this issue. Moreover, we define sets of highly related samples in terms of the…
Contrastive Self-supervised Learning (CSL) is a practical solution that learns meaningful visual representations from massive data in an unsupervised approach. The ordinary CSL embeds the features extracted from neural networks onto specific topological structures. During the training progress, the contrastive loss draws the different views of the same input together while pushing the embeddings from different inputs apart. One of the drawbacks of CSL is that the loss term requires a large number of negative samples to provide better mutual information bound ideally. However, increasing the nu…
University of Notre Dame（圣母大学）；University of Pittsburgh（匹兹堡大学）；Guangdong Provincial People’s Hospital（广东省人民医院）
The success of deep learning heavily depends on the availability of large labeled training sets. However, it is hard to get large labeled datasets in medical image domain because of the strict privacy concern and costly labeling efforts. Contrastive learning, an unsupervised learning technique, has been proved powerful in learning image-level representations from unlabeled data. The learned encoder can then be transferred or fine-tuned to improve the performance of downstream tasks with limited labels. A critical step in contrastive learning is the generation of contrastive data pairs, which i…
Temporally-Aware Supervised Contrastive Learning for Polyp Counting in Colonoscopy
Department of Mathematics, University of Padova, Padova, Italy（帕多瓦大学数学系）；Cosmo Intelligent Medical Devices, Dublin, Ireland（科莫智能医疗设备公司）；Milan Center for Neuroscience, University of Milano–Bicocca, Milan, Italy（米兰神经科学中心）
National Taiwan University（台湾大学）；Waymo LLC（Waymo有限责任公司）；University of California, Merced（加州大学默塞德分校）；Yonsei University（延世大学）；Google Research（谷歌研究院）
Self-supervised learning has recently shown great potential in vision tasks through contrastive learning, which aims to discriminate each image, or instance, in the dataset. However, such instance-level learning ignores the semantic relationship among instances and sometimes undesirably repels the anchor from the semantically similar samples, termed as "false negatives". In this work, we show that the unfavorable effect from false negatives is more significant for the large-scale datasets with more semantic concepts. To address the issue, we propose a novel self-supervised contrastive learning…
Learning Robust Representations for Malicious Content Detection via Contrastive Sampling and Uncertainty Estimation
1 College of Engineering；Computer Science, University of Central Florida, Orlando, USA, 2 Department of Computer Science \& Engineering, Mississippi State University, MS State, USA , 3 Department of Computer Science, University of Nevada, Las Vegas, USA
Shandong University（山东大学）；Joint SDU-NTU Centre for Artificial Intelligence Research (C-FAIR), Shandong University（山东大学-南洋理工大学联合人工智能研究中心（C-FAIR））；Lund University（隆德大学）；Nanyang Technological University（南洋理工大学）
Learning semantic-rich representations from raw unlabeled time series data is critical for downstream tasks such as classification and forecasting. Contrastive learning has recently shown its promising representation learning capability in the absence of expert annotations. However, existing contrastive approaches generally treat each instance independently, which leads to false negative pairs that share the same semantics. To tackle this problem, we propose MHCCL, a Masked Hierarchical Cluster-wise Contrastive Learning model, which exploits semantic information obtained from the hierarchical …
Huazhong University of Science and Technology（华中科技大学）
The rapid development of Industry 4.0 has amplified the scope and destructiveness of industrial Cyber-Physical System (CPS) by network attacks. Anomaly detection techniques are employed to identify these attacks and guarantee the normal operation of industrial CPS. However, it is still a challenging problem to cope with scenarios with few labeled samples. In this paper, we propose a few-shot anomaly detection model (FSL-PN) based on prototypical network and contrastive learning for identifying anomalies with limited labeled data from industrial CPS. Specifically, we design a contrastive loss t…
Beijing University of Posts and Telecommunications（北京邮电大学）；Hong Kong Baptist University（香港浸会大学）；Jilin University（吉林大学）
Massive false rumors emerging along with breaking news or trending topics severely hinder the truth. Existing rumor detection approaches achieve promising performance on the yesterday's news, since there is enough corpus collected from the same domain for model training. However, they are poor at detecting rumors about unforeseen events especially those propagated in different languages due to the lack of training data and prior knowledge (i.e., low-resource regimes). In this paper, we propose an adversarial contrastive learning framework to detect rumors by adapting the features learned from …
Query-based video grounding is an important yet challenging task in video understanding, which aims to localize the target segment in an untrimmed video according to a sentence query. Most previous works achieve significant progress by addressing this task in a fully-supervised manner with segment-level labels, which require high labeling cost. Although some recent efforts develop weakly-supervised methods that only need the video-level knowledge, they generally match multiple pre-defined segment proposals with query and select the best one, which lacks fine-grained frame-level details for dis…
Westlake University（西湖大学）；Westlake Institute for Advanced Study（西湖高等研究院）；Zhejiang University（浙江大学）
Contrastive Learning (CL) has emerged as a dominant technique for unsupervised representation learning which embeds augmented versions of the anchor close to each other (positive samples) and pushes the embeddings of other samples (negatives) apart. As revealed in recent studies, CL can benefit from hard negatives (negatives that are most similar to the anchor). However, we observe limited benefits when we adopt existing hard negative mining techniques of other domains in Graph Contrastive Learning (GCL). We perform both experimental and theoretical analysis on this phenomenon and find it can …
The existing SSCL of RSI is built based on constructing positive and negative sample pairs. However, due to the richness of RSI ground objects and the complexity of the RSI contextual semantics, the same RSI patches have the coexistence and imbalance of positive and negative samples, which causing the SSCL pushing negative samples far away while pushing positive samples far away, and vice versa. We call this the sample confounding issue (SCI). To solve this problem, we propose a False negAtive sampLes aware contraStive lEarning model (FALSE) for the semantic segmentation of high-resolution RSI…
Google（谷歌）；Google Research（谷歌研究院）；TTI-Chicago（芝加哥丰田技术研究所）；University of Chicago（芝加哥大学）
Self-supervised representation learning has made significant leaps fueled by progress in contrastive learning, which seeks to learn transformations that embed positive input pairs nearby, while pushing negative pairs far apart. While positive pairs can be generated reliably (e.g., as different views of the same image), it is difficult to accurately establish negative pairs, defined as samples from different images regardless of their semantic content or visual features. A fundamental problem in contrastive learning is mitigating the effects of false negatives. Contrasting false negatives induc…
Australian National University（澳大利亚国立大学）；OPPO Research Institute（OPPO研究院）；The University of Oxford（牛津大学）；Shanghai AI Lab（上海人工智能实验室）；Beihang University（北京航空航天大学）
Self-supervised audio-visual source localization aims to locate sound-source objects in video frames without extra annotations. Recent methods often approach this goal with the help of contrastive learning, which assumes only the audio and visual contents from the same video are positive samples for each other. However, this assumption would suffer from false negative samples in real-world training. For example, for an audio sample, treating the frames from the same audio class as negative samples may mislead the model and therefore harm the learned representations e.g., the audio of a siren w…
Topology Reorganized Graph Contrastive Learning with Mitigating Semantic Drift
College of Computer Science & Technology, Nanjing University of Aeronautics & Astronautics（南京航空航天大学计算机科学与技术学院）；MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, Nanjing University of Aeronautics & Astronautics（南京航空航天大学工信部模式分析与机器智能重点实验室）
Recovering binary programs' call graphs is crucial for inter-procedural analysis tasks and applications based on them.transfer One of the core challenges is recognizing targets of indirect calls (i.e., indirect callees). Existing solutions all have high false positives and negatives, making call graphs inaccurate. In this paper, we propose a new solution Callee combining transfer learning and contrastive learning. The key insight is that, deep neural networks (DNNs) can automatically identify patterns concerning indirect calls, which can be more efficient than designing approximation algorithm…
Large-scale and Fine-grained Vision-language Pre-training for Enhanced CT Image Understanding
DAMO Academy, Alibaba Group（达摩院（阿里巴巴集团））；The First Affiliated Hospital of College of Medicine, Zhejiang University（浙江大学医学院附属第一医院）；Zhejiang University（浙江大学）；Westlake University（西湖大学）
School of Computer Science and Technology, Beijing Institute of Technology（北京理工大学计算机科学与技术学院）；School of Computer Science, Beijing University of Posts and Telecommunications（北京邮电大学计算机学院）；Department of Computer Science, Tongji University（同济大学计算机科学系）；State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications（北京邮电大学网络与交换技术国家重点实验室）
Huazhong University of Science and Technology (HUST)（华中科技大学）
How to sample high quality negative instances from unlabeled data, i.e., negative sampling, is important for training implicit collaborative filtering and contrastive learning models. Although previous studies have proposed some approaches to sample informative instances, few has been done to discriminating false negative from true negative for unbiased negative sampling. On the basis of our order relation analysis of negatives' scores, we first derive the class conditional density of true negatives and that of false negatives. We next design a Bayesian classifier for negative classification, …
School of Informatics, Xiamen University, Xiamen, China（厦门大学信息学院）；National Institute for Data Science in Health and Medicine, Xiamen University（厦门大学健康医学数据科学研究院）；Jarvis Research Center, Tencent YouTu Lab, Shenzhen, China（腾讯YouTu实验室 Jarvis研究部）；Medical Artificial Intelligence Laboratory, Westlake University, Hangzhou, China（西湖大学医学人工智能实验室）
Contrastive Desensitization Learning for Cross Domain Face Forgery Detection
College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics & MIIT Key Laboratory of Pattern Analysis and Machine Intelligence（计算机科学与技术学院，南京航空航天大学及信息通信技术部模式分析与机器智能重点实验室）
