Adversarial Masking Contrastive Learning for vein recognition
Chongqing Technology and Business University（重庆工商大学）；Telecom SudParis（巴黎电信学院）；University Paris-Saclay（巴黎-萨克雷大学）；China University of Mining and Technology（中国矿业大学）
Huafeng Qin, Yiquan Wu, Mounim A. El-Yacoubi, Jun Wang, Guangxiang Yang
Vein recognition has received increasing attention due to its high security and privacy. Recently, deep neural networks such as Convolutional neural networks (CNN) and Transformers have been introduced for vein recognition and achieved state-of-the-art performance. Despite the recent advances, however, existing solutions for finger-vein feature extraction are still not optimal due to scarce training image samples. To overcome this problem, in this paper, we propose an adversarial masking contrastive learning (AMCL) approach, that generates challenging samples to train a more robust contrastive learning model for the downstream palm-vein recognition task, by alternatively optimizing the encoder in the contrastive learning model and a set of latent variables. First, a huge number of masks are generated to train a robust generative adversarial network (GAN). The trained generator transforms a latent variable from the latent variable space into a mask space. Then, we combine the trained generator with a contrastive learning model to obtain our AMCL, where the generator produces challenging masking images to increase the contrastive loss and the contrastive learning model is trained based on the harder images to learn a more robust feature representation. After training, the trained encoder in the contrastive learning model is combined with a classification layer to build a classifier, which is further fine-tuned on labeled training data for vein recognition. The experimental results on three databases demonstrate that our approach outperforms existing contrastive learning approaches in terms of improving identification accuracy of vein classifiers and achieves state-of-the-art recognition results.
Sublingual vein is commonly used to diagnose the health status. The width of main sublingual veins gives information of the blood circulation. Therefore, it is necessary to segment the main sublingual veins from the tongue automatically. In general, the dataset in the medical field is small, which is a challenge for training the deep learning model. In order to train the model with a small data set, the proposed method for automatically segmenting the sublingual veins is to re-train U-net model with different sets of the limited number of labels for the same training images. With pre-knowledge…
Forget-It-All: Multi-Concept Machine Unlearning via Concept-Aware Neuron Masking
The University of Arizona（亚利桑那大学）；The University of Tulsa（塔尔萨大学）；Clemson University（克莱姆森大学）；Western Digital Corporation（西部数据公司）；University of Georgia（佐治亚大学）
UnICLAM:Contrastive Representation Learning with Adversarial Masking for Unified and Interpretable Medical Vision Question Answering
Zhejiang University（浙江大学）；University of Illinois Urbana–Champaign Institute（伊利诺伊大学厄巴纳-香槟分校研究院）；Sixth Medical Center of Chinese PLA General Hospital（中国人民解放军总医院第六医学中心）
Hebei University of Technology（河北工业大学）；Shenzhen University（深圳大学）；TU Wien（维也纳技术大学）；University of Oxford（牛津大学）
Existing self-supervised learning methods based on contrastive learning and masked image modeling have demonstrated impressive performances. However, current masked image modeling methods are mainly utilized in natural images, and their applications in medical images are relatively lacking. Besides, their fixed high masking strategy limits the upper bound of conditional mutual information, and the gradient noise is considerable, making less the learned representation information. Motivated by these limitations, in this paper, we propose masked patches selection and adaptive masking strategy ba…
Joint Low-level and High-level Textual Representation Learning with Multiple Masking Strategies
Wenzhou University Artificial Intelligence and Advanced Manufacturing Institute (AIAMI)（温州大学人工智能与先进制造研究院）；Graduate School of Engineering, Tohoku University（东京东京大学工学研究科）
Self-supervised learning has become a popular approach in recent years for its ability to learn meaningful representations without the need for data annotation. This paper proposes a novel image augmentation technique, overlaying images, which has not been widely applied in self-supervised learning. This method is designed to provide better guidance for the model to understand underlying information, resulting in more useful representations. The proposed method is evaluated using contrastive learning, a widely used self-supervised learning method that has shown solid performance in downstream …
Tissue-Contrastive Semi-Masked Autoencoders for Segmentation Pretraining on Chest CT
Hangzhou City University（杭州城市大学）；Southwest Hospital, Army Medical University (Third Military Medical University)（陆军军医大学（第三军医大学）西南医院）；Zhejiang University（浙江大学）
Video understanding relies on perceiving the global content and modeling its internal connections (e.g., causality, movement, and spatio-temporal correspondence). To learn these interactions, we apply a mask-then-predict pre-training task on discretized video tokens generated via VQ-VAE. Unlike language, where the text tokens are more independent, neighboring video tokens typically have strong correlations (e.g., consecutive video frames usually look very similar), and hence uniformly masking individual tokens will make the task too trivial to learn useful representations. To deal with this is…
OpenUS: A Fully Open-Source Foundation Model for Ultrasound Image Analysis via Self-Adaptive Masked Contrastive Learning
Digital Environment Research Institute (DERI), Queen Mary University of London（女王玛丽大学数字环境研究 institute（DERI））；The William Harvey Research Institute, Queen Mary University of London（女王玛丽大学威廉·哈维研究 institute）；School of Electronic Engineering and Computer Science, Queen Mary University of London（女王玛丽大学电子工程与计算机科学学院）
InfMasking: Unleashing Synergistic Information by Contrastive Multimodal Interactions
School of Computing and Artificial Intelligence（计算与人工智能学院）；Engineering Research Center of Intelligent Finance Ministry of Education（教育部长智能金融工程研究中心）；Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Electronic Science and Technology of China（电子科技大学）；Shanghai Academy of AI for Science（上海人工智能科学研究院）；Artificial Intelligence Innovation and Incubation Institute Fudan University（复旦大学人工智能创新与孵化院）；Artificial Intelligence and Digital Finance Key Laboratory of Sichuan Province（四川省人工智能与数字金融重点实验室）
We study a new family of inverse problems for recovering representations of corrupted data. We assume access to a pre-trained representation learning network R(x) that operates on clean images, like CLIP. The problem is to recover the representation of an image R(x), if we are only given a corrupted version A(x), for some known forward operator A. We propose a supervised inversion method that uses a contrastive objective to obtain excellent representations for highly corrupted images. Using a linear probe on our robust representations, we achieve a higher accuracy than end-to-end supervised ba…
Attempt to fully discover the temporal diversity and chronological characteristics for self-supervised video representation learning, this work takes advantage of the temporal dependencies within videos and further proposes a novel self-supervised method named Temporal Contrastive Graph Learning (TCGL). In contrast to the existing methods that ignore modeling elaborate temporal dependencies, our TCGL roots in a hybrid graph contrastive learning strategy to jointly regard the inter-snippet and intra-snippet temporal dependencies as self-supervision signals for temporal representation learning. …
Building scalable models to learn from diverse, multimodal data remains an open challenge. For vision-language data, the dominant approaches are based on contrastive learning objectives that train a separate encoder for each modality. While effective, contrastive learning approaches introduce sampling bias depending on the data augmentations used, which can degrade performance on downstream tasks. Moreover, these methods are limited to paired image-text data, and cannot leverage widely-available unpaired data. In this paper, we investigate whether a large multimodal model trained purely via ma…
University of Stuttgart（斯图加特大学）；AISIA Lab, University of Science -VNU HCM（越南国立大学胡志明市理科大学AISIA实验室）；Dongguk University（东国大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Helmholtz AI, Helmholtz Munich（亥姆霍兹慕尼黑亥姆霍兹人工智能）；University of Bonn（波恩大学）；University of California San Diego（加利福尼亚大学圣迭戈分校）；Oldenburg University（奥尔登堡大学）；German Research Center for Artificial Intelligence（德国人工智能研究中心）
Collecting large-scale medical datasets with fully annotated samples for training of deep networks is prohibitively expensive, especially for 3D volume data. Recent breakthroughs in self-supervised learning (SSL) offer the ability to overcome the lack of labeled training samples by learning feature representations from unlabeled data. However, most current SSL techniques in the medical field have been designed for either 2D images or 3D volumes. In practice, this restricts the capability to fully leverage unlabeled data from numerous sources, which may include both 2D and 3D data. Additionally…
Cross-Modal Consistency Learning for Sign Language Recognition
University of Science and Technology of China（中国科学技术大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）；University of Texas at Austin（德克萨斯大学奥斯汀分校）
