Data Augmentation with Manifold Exploring Geometric Transformations for Increased Performance and Robustness
Technische Universität München（慕尼黑工业大学）；Ludwig-Maximilian-University, Munich（慕尼黑大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；Computer Aided Medical Procedures, Technische Universität München（慕尼黑工业大学计算机辅助医疗程序（此处按要求取到大学级别，原机构名中的部门名称保留以明确归属））；Department of Child and Adolescent Psychiatry, Psychosomatic and Psychotherapy, Ludwig-Maximilian-University, Munich（慕尼黑大学儿童青少年精神病学、心身医学与心理治疗系（此处按要求取到大学级别，原机构名中的部门名称保留以明确归属））；Computer Aided Medical Procedures, Johns Hopkins University（约翰斯·霍普金斯大学计算机辅助医疗程序（此处按要求取到大学级别，原机构名中的部门名称保留以明确归属））
Magdalini Paschali, Walter Simson, Abhijit Guha Roy, Muhammad Ferjad Naeem, Rüdiger Göbl, Christian Wachinger, Nassir Navab
备注：
Under Review for the 26th International Conference on Information Processing in Medical Imaging (IPMI) 2019
In this paper we propose a novel augmentation technique that improves not only the performance of deep neural networks on clean test data, but also significantly increases their robustness to random transformations, both affine and projective. Inspired by ManiFool, the augmentation is performed by a line-search manifold-exploration method that learns affine geometric transformations that lead to the misclassification on an image, while ensuring that it remains on the same manifold as the training data.
This augmentation method populates any training dataset with images that lie on the border of the manifolds between two-classes and maximizes the variance the network is exposed to during training. Our method was thoroughly evaluated on the challenging tasks of fine-grained skin lesion classification from limited data, and breast tumor classification of mammograms. Compared with traditional augmentation methods, and with images synthesized by Generative Adversarial Networks our method not only achieves state-of-the-art performance but also significantly improves the network's robustness.
University of Science and Technology of China（中国科学技术大学）
For machine learning task, lacking sufficient samples mean the trained model has low confidence to approach the ground truth function. Until recently, after the generative adversarial networks (GAN) had been proposed, we see the hope of small samples data augmentation (DA) with realistic fake data, and many works validated the viability of GAN-based DA. Although most of the works pointed out higher accuracy can be achieved using GAN-based DA, some researchers stressed that the fake data generated from GAN has inherent bias, and in this paper, we explored when the bias is so low that it cannot …
Tel Aviv University（特拉维夫大学）；The Chaim Sheba Medical Center（夏姆谢巴医疗中心）；Bar-Ilan University（巴伊兰大学）
Deep learning methods, and in particular convolutional neural networks (CNNs), have led to an enormous breakthrough in a wide range of computer vision tasks, primarily by using large-scale annotated datasets. However, obtaining such datasets in the medical domain remains a challenge. In this paper, we present methods for generating synthetic medical images using recently presented deep learning Generative Adversarial Networks (GANs). Furthermore, we show that generated medical images can be used for synthetic data augmentation, and improve the performance of CNN for medical image classificatio…
Due to the data shortage problem, which is one of the major problems in the field of machine learning, the accuracy level of many applications remains well below the expected. It prevents researchers from producing new artificial intelligence-based systems with the available data. This problem can be solved by generating new synthetic data with augmentation methods. In this study, new ECG signals are produced using MIT-BIH Arrhythmia Database by using Generative Adversarial Neural Networks (GAN), which is a modern data augmentation method. These generated data are used for training a machine l…
Retinal Vessel Segmentation is important for the diagnosis of various diseases. The research on retinal vessel segmentation focuses mainly on the improvement of the segmentation model which is usually based on U-Net architecture. In our study, we use the U-Net architecture and we rely on heavy data augmentation in order to achieve better performance. The success of the data augmentation relies on successfully addressing the problem of input images. By analyzing input images and performing the augmentation accordingly we show that the performance of the U-Net model can be increased dramatically…
Time-series data are one of the fundamental types of raw data representation used in data-driven techniques. In machine condition monitoring, time-series vibration data are overly used in data mining for deep neural networks. Typically, vibration data is converted into images for classification using Deep Neural Networks (DNNs), and scalograms are the most effective form of image representation. However, the DNN classifiers require huge labeled training samples to reach their optimum performance. So, many forms of data augmentation techniques are applied to the classifiers to compensate for th…
Department of Computer Science（计算机科学系）；Sheffield Institute for Translational Neuroscience (SITraN)（谢菲尔德转化神经科学研究所（SITraN））；Department of Human Communication Sciences（人类传播科学系）；Department of Neuroscience（神经科学系）；Royal Hallamshire Hospital（皇家哈勒姆郡医院）；Academic Neurology Unit, Royal Hallamshire Hospital（皇家哈勒姆郡医院学术神经科）
Data limitation is one of the most common issues in training machine learning classifiers for medical applications. Due to ethical concerns and data privacy, the number of people that can be recruited to such experiments is generally smaller than the number of participants contributing to non-healthcare datasets. Recent research showed that generative models can be used as an effective approach for data augmentation, which can ultimately help to train more robust classifiers sparse data domains. A number of studies proved that this data augmentation technique works for image and audio data set…
This paper tackles one of the greatest limitations in Machine Learning: Data Scarcity. Specifically, we explore whether high accuracy classifiers can be built from small datasets, utilizing a combination of data augmentation techniques and machine learning algorithms. In this paper, we experiment with Easy Data Augmentation (EDA) and Backtranslation, as well as with three popular learning algorithms, Logistic Regression, Support Vector Machine (SVM), and Bidirectional Long Short-Term Memory Network (Bi-LSTM). For our experimentation, we utilize the Wikipedia Toxic Comments dataset so that in t…
Nanyang Technological University（南洋理工大学）；China-Singapore International Joint Research Institute（中新国际联合研究院）；Sun Yat-sen University（中山大学）；The University of Newcastle（纽卡斯尔大学）
Code search, which aims at retrieving the most relevant code fragment for a given natural language query, is a common activity in software development practice. Recently, contrastive learning is widely used in code search research, where many data augmentation approaches for source code (e.g., semantic-preserving program transformation) are proposed to learn better representations. However, these augmentations are at the raw-data level, which requires additional code analysis in the preprocessing stage and additional training costs in the training stage. In this paper, we explore augmentation …
Exploring specialization and sensitivity of convolutional neural networks in the context of simultaneous image augmentations
Marchuk Institute of Numerical Mathematics（马尔库克数值数学研究所）；Skolkovo Institute of Science and Technology（斯克尔科沃科学与技术研究所）；Faculty of Computational Mathematics and Cybernetics, Lomonosov Moscow State University（洛蒙诺索夫莫斯科国立大学计算数学与 cybernetics 学院）；Artificial Intelligence Research Institute (AIRI)（人工智能研究 institute (AIRI)）
Enhancing machine capabilities to answer questions has been a topic of considerable focus in recent years of NLP research. Language models like Embeddings from Language Models (ELMo)[1] and Bidirectional Encoder Representations from Transformers (BERT) [2] have been very successful in developing general purpose language models that can be optimized for a large number of downstream language tasks. In this work, we focused on augmenting the pre-trained BERT language model with different output neural net architectures and compared their performance on question answering task posed by the Stanfor…
Exploring Data Augmentation and Resampling Strategies for Transformer-Based Models to Address Class Imbalance in AI Scoring of Scientific Explanations in NGSS Classroom
Michigan State University（密歇根州立大学）；CREATE for STEM（STEM创新计划）；University of Houston（休斯顿大学）
Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）
While directly fine-tuning (FT) large-scale, pretrained models on task-specific data is well-known to induce strong in-distribution task performance, recent works have demonstrated that different adaptation protocols, such as linear probing (LP) prior to FT, can improve out-of-distribution generalization. However, the design space of such adaptation protocols remains under-explored and the evaluation of such protocols has primarily focused on distribution shifts. Therefore, in this work, we evaluate common adaptation protocols across distributions shifts and machine learning safety metrics (e.…
In this work, we consider the problem of imbalanced data in a regression framework when the imbalanced phenomenon concerns continuous or discrete covariates. Such a situation can lead to biases in the estimates. In this case, we propose a data augmentation algorithm that combines a weighted resampling (WR) and a data augmentation (DA) procedure. In a first step, the DA procedure permits exploring a wider support than the initial one. In a second step, the WR method drives the exogenous distribution to a target one. We discuss the choice of the DA procedure through a numerical study that illust…
Exploring adversarial attack vectors and studying their effects on machine learning algorithms has been of interest to researchers. Deep neural networks working with time series data have received lesser interest compared to their image counterparts in this context. In a recent finding, it has been revealed that current state-of-the-art deep learning time series classifiers are vulnerable to adversarial attacks. In this paper, we introduce two local gradient based and one spectral density based time series data augmentation techniques. We show that a model trained with data obtained using our …
With the world population projected to near 10 billion by 2050, minimizing crop damage and guaranteeing food security has never been more important. Machine learning has been proposed as a solution to quickly and efficiently identify diseases in crops. Convolutional Neural Networks typically require large datasets of annotated data which are not available on demand. Collecting this data is a long and arduous process which involves manually picking, imaging, and annotating each individual leaf. I tackle the problem of plant image data scarcity by exploring the efficacy of various data augmentat…
Automatic brain tumor segmentation from Magnetic Resonance Imaging (MRI) data plays an important role in assessing tumor response to therapy and personalized treatment stratification.Manual segmentation is tedious and subjective.Deep-learning-based algorithms for brain tumor segmentation have the potential to provide objective and fast tumor segmentation.However, the training of such algorithms requires large datasets which are not always available. Data augmentation techniques may reduce the need for large datasets.However current approaches are mostly parametric and may result in suboptimal …
University of Toronto（多伦多大学）；Sunnybrook Research Institute（桑尼布鲁克研究所）
The segmentation of the breast from the chest wall is an important first step in the analysis of breast magnetic resonance images. 3D U-nets have been shown to obtain high segmentation accuracy and appear to generalize well when trained on one scanner type and tested on another scanner, provided that a very similar T1-weighted MR protocol is used. There has, however, been little work addressing the problem of domain adaptation when image intensities or patient orientation differ markedly between the training set and an unseen test set. To overcome the domain shift we propose to apply extensive…
Fast Convergence through Distributed Augmentation for Class-Imbalanced Federated Learning
Amar Nath and Shashi Khosla School of Information Technology, Indian Institute of Technology Delhi（印度理工学院德里分校 Amar Nath and Shashi Khosla 信息技术学院）；Department of Electrical Engineering, Indian Institute of Technology Delhi（印度理工学院德里分校电气工程系）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）
University of Electronic Science and Technology of China（电子科技大学）；University of Toronto（多伦多大学）；The Hospital for Sick Children (SickKids)（SickKids 儿童医院（病童医院））；Sinai Health System（西奈卫生系统）；University Health Network（大学健康网络）；Lunenfeld-Tanenbaum Research Institute（卢嫩菲尔德-塔南鲍姆研究所）；Sunnybrook Research Institute（桑尼布鲁克研究所）
Data augmentation refers to a group of techniques whose goal is to battle limited amount of available data to improve model generalization and push sample distribution toward the true distribution. While different augmentation strategies and their combinations have been investigated for various computer vision tasks in the context of deep learning, a specific work in the domain of medical imaging is rare and to the best of our knowledge, there has been no dedicated work on exploring the effects of various augmentation methods on the performance of deep learning models in prostate cancer detect…
Vision Transformers (ViT) have been shown to attain highly competitive performance for a wide range of vision applications, such as image classification, object detection and semantic image segmentation. In comparison to convolutional neural networks, the Vision Transformer's weaker inductive bias is generally found to cause an increased reliance on model regularization or data augmentation ("AugReg" for short) when training on smaller training datasets. We conduct a systematic empirical study in order to better understand the interplay between the amount of training data, AugReg, model size a…
University of California, Los Angeles（加州大学洛杉矶分校）；Johns Hopkins University（约翰斯·霍普金斯大学）
The effects of speaking-style variability on automatic speaker verification were investigated using the UCLA Speaker Variability database which comprises multiple speaking styles per speaker. An x-vector/PLDA (probabilistic linear discriminant analysis) system was trained with the SRE and Switchboard databases with standard augmentation techniques and evaluated with utterances from the UCLA database. The equal error rate (EER) was low when enrollment and test utterances were of the same style (e.g., 0.98% and 0.57% for read and conversational speech, respectively), but it increased substantial…
Snap Inc.（Snap公司）；University of Notre Dame（圣母大学）；Michigan State University（密歇根州立大学）；Cornell University（康奈尔大学）；Technical University of Munich（慕尼黑工业大学）
Data augmentation has recently seen increased interest in graph machine learning given its demonstrated ability to improve model performance and generalization by added training data. Despite this recent surge, the area is still relatively under-explored, due to the challenges brought by complex, non-Euclidean structure of graph data, which limits the direct analogizing of traditional augmentation operations on other types of image, video or text data. Our work aims to give a necessary and timely overview of existing graph data augmentation methods; notably, we present a comprehensive and syst…
University of Science and Technology of China（中国科学技术大学）；JD Explore Academy（京东探索研究院）；The University of Sydney（悉尼大学）
Data-Efficient GANs (DE-GANs), which aim to learn generative models with a limited amount of training data, encounter several challenges for generating high-quality samples. Since data augmentation strategies have largely alleviated the training instability, how to further improve the generative performance of DE-GANs becomes a hotspot. Recently, contrastive learning has shown the great potential of increasing the synthesis quality of DE-GANs, yet related principles are not well explored. In this paper, we revisit and compare different contrastive learning strategies in DE-GANs, and identify (…
Labeling semantic segmentation datasets is a costly and laborious process if compared with tasks like image classification and object detection. This is especially true for remote sensing applications that not only work with extremely high spatial resolution data but also commonly require the knowledge of experts of the area to perform the manual labeling. Data augmentation techniques help to improve deep learning models under the circumstance of few and imbalanced labeled samples. In this work, we propose a novel data augmentation method focused on exploring the spatial context of remote sens…
The study of adversarial robustness has so far largely focused on perturbations bound in p-norms. However, state-of-the-art models turn out to be also vulnerable to other, more natural classes of perturbations such as translations and rotations. In this work, we thoroughly investigate the vulnerability of neural network--based classifiers to rotations and translations. While data augmentation offers relatively small robustness, we use ideas from robust optimization and test-time input aggregation to significantly improve robustness. Finally we find that, in contrast to the p-norm case, first-o…
Supervised training of neural networks requires large, diverse and well annotated data sets. In the medical field, this is often difficult to achieve due to constraints in time, expert knowledge and prevalence of an event. Artificial data augmentation can help to prevent overfitting and improve the detection of rare events as well as overall performance. However, most augmentation techniques use purely spatial transformations, which are not sufficient for video data with temporal correlations. In this paper, we present a novel methodology for workflow augmentation and demonstrate its benefit f…
Exploring Geographic Relative Space in Large Language Models through Activation Patching
School of Geography, Geology and the Environment, University of Leicester, UK（地理、地质与环境学院，莱斯特大学，英国）；Department of Mathematics, Computer Science and Physics, University of Udine, Italy（数学、计算机科学与物理系，乌迪内大学，意大利）
Offline reinforcement learning (Offline RL) suffers from the innate distributional shift as it cannot interact with the physical environment during training. To alleviate such limitation, state-based offline RL leverages a learned dynamics model from the logged experience and augments the predicted state transition to extend the data distribution. For exploiting such benefit also on the image-based RL, we firstly propose a generative model, S2P (State2Pixel), which synthesizes the raw pixel of the agent from its corresponding state. It enables bridging the gap between the state and the image d…
IBM T. J. Watson Research Center（IBM T. J. 沃森研究中心）
Increased availability of electronic health records (EHR) has enabled researchers to study various medical questions. Cohort selection for the hypothesis under investigation is one of the main consideration for EHR analysis. For uncommon diseases, cohorts extracted from EHRs contain very limited number of records - hampering the robustness of any analysis. Data augmentation methods have been successfully applied in other domains to address this issue mainly using simulated records. In this paper, we present ODVICE, a data augmentation framework that leverages the medical concept ontology to sy…
Siamese networks have become a common structure in various recent models for unsupervised visual representation learning. These models maximize the similarity between two augmentations of one image, subject to certain conditions for avoiding collapsing solutions. In this paper, we report surprising empirical results that simple Siamese networks can learn meaningful representations even using none of the following: (i) negative sample pairs, (ii) large batches, (iii) momentum encoders. Our experiments show that collapsing solutions do exist for the loss and structure, but a stop-gradient operat…
Most EEG-based Brain-Computer Interfaces (BCIs) require a considerable amount of training data to calibrate the classification model, owing to the high variability in the EEG data, which manifests itself between participants, but also within participants from session to session (and, of course, from trial to trial). In general, the more complex the model, the more data for training is needed. We suggest a novel technique for augmenting the training data by generating new data from the data set at hand. Different from existing techniques, our method uses backward and forward projection using so…
Carnegie Mellon University（卡内基梅隆大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Colorado Denver - Anschutz Medical Campus（科罗拉多大学丹佛分校安舒茨医学校区）；Allegheny Health Network（阿勒格尼健康网络）
There has been an increased interest in applying deep neural networks to automatically interpret and analyze the 12-lead electrocardiogram (ECG). The current paradigms with machine learning methods are often limited by the amount of labeled data. This phenomenon is particularly problematic for clinically-relevant data, where labeling at scale can be time-consuming and costly in terms of the specialized expertise and human effort required. Moreover, deep learning classifiers may be vulnerable to adversarial examples and perturbations, which could have catastrophic consequences, for example, whe…
Recently, vision transformers and MLP-based models have been developed in order to address some of the prevalent weaknesses in convolutional neural networks. Due to the novelty of transformers being used in this domain along with the self-attention mechanism, it remains unclear to what degree these architectures are robust to corruptions. Despite some works proposing that data augmentation remains essential for a model to be robust against corruptions, we propose to explore the impact that the architecture has on corruption robustness. We find that vision transformer architectures are inherent…
Data augmentation has become a de facto component for training high-performance deep image classifiers, but its potential is under-explored for object detection. Noting that most state-of-the-art object detectors benefit from fine-tuning a pre-trained classifier, we first study how the classifiers' gains from various data augmentations transfer to object detection. The results are discouraging; the gains diminish after fine-tuning in terms of either accuracy or robustness. This work instead augments the fine-tuning stage for object detectors by exploring adversarial examples, which can be view…
Carnegie Mellon University（卡内基梅隆大学）；Google Research（谷歌研究院）；Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）；Dartmouth College（达特茅斯学院）
Data augmentation has recently seen increased interest in NLP due to more work in low-resource domains, new tasks, and the popularity of large-scale neural networks that require large amounts of training data. Despite this recent upsurge, this area is still relatively underexplored, perhaps due to the challenges posed by the discrete nature of language data. In this paper, we present a comprehensive and unifying survey of data augmentation for NLP by summarizing the literature in a structured manner. We first introduce and motivate data augmentation for NLP, and then discuss major methodologic…
College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机与软件学院）；Department of Computing, The Hong Kong Polytechnic University（香港理工大学电子计算学系）
The data scarcity problem in Electroencephalography (EEG) based affective computing results into difficulty in building an effective model with high accuracy and stability using machine learning algorithms especially deep learning models. Data augmentation has recently achieved considerable performance improvement for deep learning models: increased accuracy, stability, and reduced over-fitting. In this paper, we propose a novel data augmentation framework, namely Generative Adversarial Network-based Self-supervised Data Augmentation (GANSER). As the first to combine adversarial training with …
