Supervised training of an automated medical image analysis system often requires a large amount of expert annotations that are hard to collect. Moreover, the proportions of data available across different classes may be highly imbalanced for rare diseases. To mitigate these issues, we investigate a novel data augmentation pipeline that selectively adds new synthetic images generated by conditional Adversarial Networks (cGANs), rather than extending directly the training set with synthetic images. The selection mechanisms that we introduce to the synthetic augmentation pipeline are motivated by the observation that, although cGAN-generated images can be visually appealing, they are not guaranteed to contain essential features for classification performance improvement. By selecting synthetic images based on the confidence of their assigned labels and their feature similarity to real labeled images, our framework provides quality assurance to synthetic augmentation by ensuring that adding the selected synthetic images to the training set will improve performance. We evaluate our model on a medical histopathology dataset, and two natural image classification benchmarks, CIFAR10 and SVHN. Results on these datasets show significant and consistent improvements in classification performance (with 6.8%, 3.9%, 1.6% higher accuracy, respectively) by leveraging cGAN generated images with selective augmentation.
The Pennsylvania State University（宾夕法尼亚州立大学）；Lister Hill National Center for Biomedical Communications（利斯特希尔国家生物医学通信中心）；National Library of Medicine（国家医学图书馆）；Penn State Health Milton S. Hershey Medical Center（宾夕法尼亚州立大学健康米尔顿·S·赫尔希医学中心）；Penn State College Of Medicine（宾夕法尼亚州立大学医学院）
Histopathological analysis is the present gold standard for precancerous lesion diagnosis. The goal of automated histopathological classification from digital images requires supervised training, which requires a large number of expert annotations that can be expensive and time-consuming to collect. Meanwhile, accurate classification of image patches cropped from whole-slide images is essential for standard sliding window based histopathology slide classification methods. To mitigate these issues, we propose a carefully designed conditional GAN model, namely HistoGAN, for synthesizing realisti…
Universal Lymph Node Detection in Multiparametric MRI with Selective Augmentation
National Institutes of Health（美国国立卫生研究院）；National Library of Medicine（美国国家医学图书馆）；National Center for Biotechnology Information（国家生物技术信息中心）；Clinical Center（临床中心）；Radiology and Imaging Sciences（放射学与影像科学科）
We present a deep person re-identification approach that combines semantically selective, deep data augmentation with clustering-based network compression to generate high performance, light and fast inference networks. In particular, we propose to augment limited training data via sampling from a deep convolutional generative adversarial network (DCGAN), whose discriminator is constrained by a semantic classifier to explicitly control the domain specificity of the generation process. Thereby, we encode information in the classifier network which can be utilized to steer adversarial synthesis,…
Recent self-supervised video representation learning methods focus on maximizing the similarity between multiple augmented views from the same video and largely rely on the quality of generated views. However, most existing methods lack a mechanism to prevent representation learning from bias towards static information in the video. In this paper, we propose frequency augmentation (FreqAug), a spatio-temporal data augmentation method in the frequency domain for video representation learning. FreqAug stochastically removes specific frequency components from the video so that learned representat…
Interpretability of deep learning (DL) systems is gaining attention in medical imaging to increase experts' trust in the obtained predictions and facilitate their integration in clinical settings. We propose a deep visualization method to generate interpretability of DL classification tasks in medical imaging by means of visual evidence augmentation. The proposed method iteratively unveils abnormalities based on the prediction of a classifier trained only with image-level labels. For each image, initial visual evidence of the prediction is extracted with a given visual attribution technique. T…
This paper presents the selective use of eye-gaze information in learning human actions in Atari games. Vast evidence suggests that our eye movement convey a wealth of information about the direction of our attention and mental states and encode the information necessary to complete a task. Based on this evidence, we hypothesize that selective use of eye-gaze, as a clue for attention direction, will enhance the learning from demonstration. For this purpose, we propose a selective eye-gaze augmentation (SEA) network that learns when to use the eye-gaze information. The proposed network architec…
Harbin Institute of Technology（哈尔滨工业大学）；Tianjin University（天津大学）
Collection of massive well-annotated samples is effective in improving object detection performance but is extremely laborious and costly. Instead of data collection and annotation, the recently proposed Cut-Paste methods [12, 15] show the potential to augment training dataset by cutting foreground objects and pasting them on proper new backgrounds. However, existing Cut-Paste methods cannot guarantee synthetic images always precisely model visual context, and all of them require external datasets. To handle above issues, this paper proposes a simple yet effective instance-switching (IS) strat…
Eurecat, Centre Tecnològic de Catalunya（加泰罗尼亚技术中心Eurecat）；Computer Vision Center（计算机视觉中心）；Universitat Autònoma de Barcelona（巴塞罗那自治大学）
This paper explores the possibilities of image style transfer applied to text maintaining the original transcriptions. Results on different text domains (scene text, machine printed text and handwritten text) and cross modal results demonstrate that this is feasible, and open different research lines. Furthermore, two architectures for selective style transfer, which means transferring style to only desired image pixels, are proposed. Finally, scene text selective style transfer is evaluated as a data augmentation technique to expand scene text detection datasets, resulting in a boost of text …
One of the biggest problems in neural learning networks is the lack of training data available to train the network. Data augmentation techniques over the past few years, have therefore been developed, aiming to increase the amount of artificial training data with the limited number of real world samples. In this paper, we look particularly at the MNIST handwritten dataset an image dataset used for digit recognition, and the methods of data augmentation done on this data set. We then take a detailed look into one of the most popular augmentation techniques used for this data set elastic deform…
College of Information Sciences and Technology, The Pennsylvania State University（宾夕法尼亚州立大学信息科学与技术学院）；National Library of Medicine, National Institutes of Health（美国国立卫生研究院国家医学图书馆）；College of Medicine, The Pennsylvania State University（宾夕法尼亚州立大学医学院）
Synthesizing realistic medical images provides a feasible solution to the shortage of training data in deep learning based medical image recognition systems. However, the quality control of synthetic images for data augmentation purposes is under-investigated, and some of the generated images are not realistic and may contain misleading features that distort data distribution when mixed with real images. Thus, the effectiveness of those synthetic images in medical image recognition systems cannot be guaranteed when they are being added randomly without quality assurance. In this work, we propo…
MEDDAP: Medical Dataset Enhancement via Diversified Augmentation Pipeline
School of Biomedical Engineering, University of British Columbia（不列颠哥伦比亚大学生物医学工程学院）；BC Cancer Research Institute（不列颠哥伦比亚癌症研究所）；Department of Radiology, University of British Columbia（不列颠哥伦比亚大学放射学系）；Department of Physics, University of British Columbia（不列颠哥伦比亚大学物理系）；Department of Medicine, University of British Columbia（不列颠哥伦比亚大学医学系）
Transformer with Selective Shuffled Position Embedding and Key-Patch Exchange Strategy for Early Detection of Knee Osteoarthritis
IDP Institute（IDP研究所）；UMR CNRS 7013（法国国家科学研究中心UMR 7013）；University of Orleans（奥尔良大学）；PRISME Laboratory（PRISME实验室）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Geneva University Hospital（日内瓦大学医院）
Most modern approaches for domain adaptive semantic segmentation rely on continued access to source data during adaptation, which may be infeasible due to computational or privacy constraints. We focus on source-free domain adaptation for semantic segmentation, wherein a source model must adapt itself to a new target domain given only unlabeled target data. We propose Augmentation Consistency-guided Self-training (AUGCO), a source-free adaptation algorithm that uses the model's pixel-level predictive consistency across diverse, automatically generated views of each target image along with mode…
University of Southern California（南加州大学）；Microsoft Research（微软研究院）
We propose EM-PASTE: an Expectation Maximization(EM) guided Cut-Paste compositional dataset augmentation approach for weakly-supervised instance segmentation using only image-level supervision. The proposed method consists of three main components. The first component generates high-quality foreground object masks. To this end, an EM-like approach is proposed that iteratively refines an initial set of object mask proposals generated by a generic region proposal method. Next, in the second component, high-quality context-aware background images are generated using a text-to-image compositional …
PAUL: Uncertainty-Guided Partition and Augmentation for Robust Cross-View Geo-Localization under Noisy Correspondence
National University of Defense Technology（国防科技大学）；Shenzhen Loop Area Institute（深圳河套学院）；School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen（香港中文大学深圳人工智能学院）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学深圳研究院）
Learning Semantic Directions for Feature Augmentation in Domain-Generalized Medical Segmentation
Chengdu Institute of Compute Application, Chinese Academy of Sciences, Chengdu, China（成都计算应用研究所，中国科学院，中国）；China Zhenhua Research Institute Co., Ltd., Guiyang, China（中国振华研究院有限公司，贵阳，中国）；School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing, China（计算机科学与技术学院，中国科学院大学，北京，中国）
CrossFuse: Learning Infrared and Visible Image Fusion by Cross-Sensor Top-K Vision Alignment and Beyond
School of Information Engineering, Guangdong University of Technology（广东工业大学信息工程学院）；CRRC Academy Co., Ltd.（中车研究院有限公司）；School of Computer Science, Sun Yat-sen University（中山大学计算机学院）
Key-Exchange Convolutional Auto-Encoder for Data Augmentation in Early Knee Osteoarthritis Detection
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；University Sorbonne Paris Nord（巴黎北部索邦大学）；Jiangsu Institute of Hematology（江苏血液学研究所）；The First Affiliated Hospital of Soochow University（苏州大学附属第一医院）；Henan University of Chinese Medicine（河南中医药大学）；German Cancer Research Center（德国癌症研究中心）；Athinoula A. Martinos Centre for Biomedical Imaging（阿西努拉·A.马蒂诺斯生物医学成像中心）；Geneva University Hospital（日内瓦大学医院）；IDP Institute（IDP研究所）；University of Orleans（奥尔良大学）
Multi-task Learning (MTL) for classification with disjoint datasets aims to explore MTL when one task only has one labeled dataset. In existing methods, for each task, the unlabeled datasets are not fully exploited to facilitate this task. Inspired by semi-supervised learning, we use unlabeled datasets with pseudo labels to facilitate each task. However, there are two major issues: 1) the pseudo labels are very noisy; 2) the unlabeled datasets and the labeled dataset for each task has considerable data distribution mismatch. To address these issues, we propose our MTL with Selective Augmentati…
