Collecting fine-grained labels usually requires expert-level domain knowledge and is prohibitive to scale up. In this paper, we propose Attribute Mix, a data augmentation strategy at attribute level to expand the fine-grained samples. The principle lies in that attribute features are shared among fine-grained sub-categories, and can be seamlessly transferred among images. Toward this goal, we propose an automatic attribute mining approach to discover attributes that belong to the same super-category, and Attribute Mix is operated by mixing semantically meaningful attribute features from two images. Attribute Mix is a simple but effective data augmentation strategy that can significantly improve the recognition performance without increasing the inference budgets. Furthermore, since attributes can be shared among images from the same super-category, we further enrich the training samples with attribute level labels using images from the generic domain. Experiments on widely used fine-grained benchmarks demonstrate the effectiveness of our proposed method.
Vehicle re-identification is one of the core technologies of intelligent transportation systems and smart cities, but large intra-class diversity and inter-class similarity poses great challenges for existing method. In this paper, we propose a multi-guided learning approach which utilizing the information of attributes and meanwhile introducing two novel random augments to improve the robustness during training. What's more, we propose an attribute constraint method and group re-ranking strategy to refine matching results. Our method achieves mAP of 66.83% and rank-1 accuracy 76.05% in the CV…
Data augmentation has been highly effective in narrowing the data gap and reducing the cost for human annotation, especially for tasks where ground truth labels are difficult and expensive to acquire. In face recognition, large pose and illumination variation of face images has been a key factor for performance degradation. However, human annotation for the various face understanding tasks including face landmark localization, face attributes classification and face recognition under these challenging scenarios are highly costly to acquire. Therefore, it would be desirable to perform data augm…
National Technical University of Athens（雅典国立技术大学）；Queen Mary University of London（伦敦玛丽女王大学）
Automatic Facial Expression Recognition (FER) has attracted increasing attention in the last 20 years since facial expressions play a central role in human communication. Most FER methodologies utilize Deep Neural Networks (DNNs) that are powerful tools when it comes to data analysis. However, despite their power, these networks are prone to overfitting, as they often tend to memorize the training data. What is more, there are not currently a lot of in-the-wild (i.e. in unconstrained environment) large databases for FER. To alleviate this issue, a number of data augmentation techniques have be…
Fu Jen Catholic University（辅仁大学）；National Tsing Hua University（国立清华大学）；Tsinghua University（清华大学）
The performance of a convolutional neural network (CNN) based face recognition model largely relies on the richness of labelled training data. Collecting a training set with large variations of a face identity under different poses and illumination changes, however, is very expensive, making the diversity of within-class face images a critical issue in practice. In this paper, we propose a 3D model-assisted domain-transferred face augmentation network (DotFAN) that can generate a series of variants of an input face based on the knowledge distilled from existing rich face datasets collected fro…
Herein, we present a system for hyperspectral image segmentation that utilizes multiple class--based denoising autoencoders which are efficiently trained. Moreover, we present a novel hyperspectral data augmentation method for labelled HSI data using linear mixtures of pixels from each class, which helps the system with edge pixels which are almost always mixed pixels. Finally, we utilize a deep neural network and morphological hole-filling to provide robust image classification. Results run on the Salinas dataset verify the high performance of the proposed algorithm.
SGD-Mix: Enhancing Domain-Specific Image Classification with Label-Preserving Data Augmentation
Department of Computer Science and Information Engineering, National Cheng Kung University（计算机科学与信息工程系，国立成功大学）；Department of Biomedical Informatics, Harvard Medical School, Harvard University（生物医学信息学系，哈佛医学院，哈佛大学）
Test-Time Mixup Augmentation for Data and Class-Specific Uncertainty Estimation in Deep Learning Image Classification
School of Electrical Engineering（电气工程学院）；Korea Advanced Institute of Science and Technology（韩国科学技术院）；Department of Software Convergence（软件融合系）；College of Interdisciplinary Studies for Emerging Industries（新兴产业跨学科研究学院）；Seoul Women’s University（首尔女子大学）
MoireMix: A Formula-Based Data Augmentation for Improving Image Classification Robustness
National Institute of Advanced Industrial Science and Technology (AIST)（国立研究开发法人产业技术综合研究所）；Tokyo Denki University（东京电机大学）；Waseda University（早稻田大学）；Fundamental AI Lab, University of Technology Nuremberg（纽伦堡工业大学基础人工智能实验室）；Visual Geometry Group, University of Oxford（牛津大学视觉几何组）
UC San Diego（加州大学圣迭戈分校）；University of Salzburg（萨尔茨堡大学）；UNC Chapel Hill（北卡罗来纳大学教堂山分校）
We consider the problem of data augmentation, i.e., generating artificial samples to extend a given corpus of training data. Specifically, we propose attributed-guided augmentation (AGA) which learns a mapping that allows to synthesize data such that an attribute of a synthesized sample is at a desired value or strength. This is particularly interesting in situations where little data with no attribute annotation is available for learning, but we have access to a large external corpus of heavily annotated samples. While prior works primarily augment in the space of images, we propose to perfor…
Pedestrian attribute recognition is an important multi-label classification problem. Although the convolutional neural networks are prominent in learning discriminative features from images, the data imbalance in multi-label setting for fine-grained tasks remains an open problem. In this paper, we propose a new re-sampling algorithm called: data augmentation imbalance (DAI) to explicitly enhance the ability to discriminate the fewer attributes via increasing the proportion of labels accounting for a small part. Fundamentally, by applying over-sampling and under-sampling on the multi-label data…
School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；Department of Engineering Science, University of Oxford（牛津大学工程科学系）；School of Engineering, Newcastle University（纽卡斯尔大学工程学院）
The recent advance in deep generative models outlines a promising perspective in the realm of Zero-Shot Learning (ZSL). Most generative ZSL methods use category semantic attributes plus a Gaussian noise to generate visual features. After generating unseen samples, this family of approaches effectively transforms the ZSL problem into a supervised classification scheme. However, the existing models use a single semantic attribute, which contains the complete attribute information of the category. The generated data also carry the complete attribute information, but in reality, visual samples usu…
Deep Convolutional Neural Networks have made an incredible progress in many Computer Vision tasks. This progress, however, often relies on the availability of large amounts of the training data, required to prevent over-fitting, which in many domains entails significant cost of manual data labeling. An alternative approach is application of data augmentation (DA) techniques that aim at model regularization by creating additional observations from the available ones. This survey focuses on two DA research streams: image mixing and automated selection of augmentation strategies. First, the prese…
Raven's Progressive Matrices (RPMs) are frequently used in testing human's visual reasoning ability. Recent advances of RPM-like datasets and solution models partially address the challenges of visually understanding the RPM questions and logically reasoning the missing answers. In view of the poor generalization performance due to insufficient samples in RPM datasets, we propose an effective scheme, namely Candidate Answer Morphological Mixup (CAM-Mix). CAM-Mix serves as a data augmentation strategy by gray-scale image morphological mixup, which regularizes various solution methods and overco…
Mixup is a commonly adopted data augmentation technique for image classification. Recent advances in mixup methods primarily focus on mixing based on saliency. However, many saliency detectors require intense computation and are especially burdensome for parameter-heavy transformer models. To this end, we propose TokenMixup, an efficient attention-guided token-level data augmentation method that aims to maximize the saliency of a mixed set of tokens. TokenMixup provides x15 faster saliency-aware data augmentation compared to gradient-based methods. Moreover, we introduce a variant of TokenMixu…
Shanghai Jiao Tong University（上海交通大学）；Shanghai United Imaging Intelligence Co., Ltd.（上海联影智能医疗科技有限公司）；ShanghaiTech University（上海科技大学）
Lung nodule detection in chest X-ray (CXR) images is common to early screening of lung cancers. Deep-learning-based Computer-Assisted Diagnosis (CAD) systems can support radiologists for nodule screening in CXR. However, it requires large-scale and diverse medical data with high-quality annotations to train such robust and accurate CADs. To alleviate the limited availability of such datasets, lung nodule synthesis methods are proposed for the sake of data augmentation. Nevertheless, previous methods lack the ability to generate nodules that are realistic with the size attribute desired by the …
This paper presents a supervised mixing augmentation method termed SuperMix, which exploits the salient regions within input images to construct mixed training samples. SuperMix is designed to obtain mixed images rich in visual features and complying with realistic image priors. To enhance the efficiency of the algorithm, we develop a variant of the Newton iterative method, $65\times$ faster than gradient descent on this problem. We validate the effectiveness of SuperMix through extensive evaluations and ablation studies on two tasks of object classification and knowledge distillation. On the …
University of Engineering and Technology（工程技术大学）；King Fahad University of Petroleum and Minerals（法赫德国王石油矿产大学）；Monash University（莫纳什大学）；University of Western Australia（西澳大学）
Neural networks are prone to overfitting and memorizing data patterns. To avoid over-fitting and enhance their generalization and performance, various methods have been suggested in the literature, including dropout, regularization, label smoothing, etc. One such method is augmentation which introduces different types of corruption in the data to prevent the model from overfitting and to memorize patterns present in the data. A sub-area of data augmentation is image mixing and deleting. This specific type of augmentation either deletes image regions or mixes two images to hide or make particul…
Graduate School of Engineering（研究生院）；University of Fukui（福井大学）
In the research field of activity recognition, although it is difficult to collect a large amount of measured sensor data, there has not been much discussion about data augmentation (DA). In this study, I propose Octave Mix as a new synthetic-style DA method for sensor-based activity recognition. Octave Mix is a simple DA method that combines two types of waveforms by intersecting low and high frequency waveforms using frequency decomposition. In addition, I propose a DA ensemble model and its training algorithm to acquire robustness to the original sensor data while remaining a wide variety o…
CP2M: Clustered-Patch-Mixed Mosaic Augmentation for Aerial Image Segmentation
Northwestern University（西北大学）；Carnegie Mellon University（卡内基梅隆大学）；The University of Hong Kong（香港大学）；Duke University（杜克大学）；University College Dublin（都柏林大学学院）；Beijing University of Technology（北京工业大学）；The ADAPT SFI Research Centre（ADAPT SFI研究中心）
ReMA: A Training-Free Plug-and-Play Mixing Augmentation for Video Behavior Recognition
University of Science and Technology of China（科学技术大学）；Hefei University of Technology（合肥工业大学）；Hefei Xiaosheng Intelligent Technology Co., Ltd.（合肥小生智能科技有限公司）；Cylingo Group（Cylingo集团）；The University of Electro-Communications（电通大学）
Augmenting data in image space (eg. flipping, cropping etc) and activation space (eg. dropout) are being widely used to regularise deep neural networks and have been successfully applied on several computer vision tasks. Unlike previous works, which are mostly focused on doing augmentation in the aforementioned domains, we propose to do augmentation in label space. In this paper, we present a novel method to generate fixed dimensional labels with continuous values for images by exploiting the word2vec representations of the existing categorical labels. We then append these representations with…
Deep learning-based methods have achieved considerable success on single image dehazing in recent years. However, these methods are often subject to performance degradation when domain shifts are confronted. Specifically, haze density gaps exist among the existing datasets, often resulting in poor performance when these methods are tested across datasets. To address this issue, we propose a density-aware mixup augmentation (DAMix). DAMix generates samples in an attempt to minimize the Wasserstein distance with the hazy images in the target domain. These DAMix-ed samples not only mitigate domai…
Tarbiat Modares University（塔比阿特莫达雷斯大学）；NRP company（NRP公司）；University of Warwick（华威大学）；University of Sheffield（谢菲尔德大学）
Computer-aided diagnosis systems for classification of different type of skin lesions have been an active field of research in recent decades. It has been shown that introducing lesions and their attributes masks into lesion classification pipeline can greatly improve the performance. In this paper, we propose a framework by incorporating transfer learning for segmenting lesions and their attributes based on the convolutional neural networks. The proposed framework is based on the encoder-decoder architecture which utilizes a variety of pre-trained networks in the encoding path and generates t…
Data augmentation is an effective regularization strategy to alleviate the overfitting, which is an inherent drawback of the deep neural networks. However, data augmentation is rarely considered for point cloud processing despite many studies proposing various augmentation methods for image data. Actually, regularization is essential for point clouds since lack of generality is more likely to occur in point cloud due to small datasets. This paper proposes a Rigid Subset Mix (RSMix), a novel data augmentation method for point clouds that generates a virtual mixed sample by replacing part of the…
Seoul National University（首尔国立大学）；Chung-Ang University（中央大学）；NAVER Clova
Image-mixing augmentations (e.g., Mixup and CutMix), which typically involve mixing two images, have become the de-facto training techniques for image classification. Despite their huge success in image classification, the number of images to be mixed has not been elucidated in the literature: only the naive K-image expansion has been shown to lead to performance degradation. This study derives a new K-image mixing augmentation based on the stick-breaking process under Dirichlet prior distribution. We demonstrate the superiority of our K-image expansion augmentation over conventional two-image…
In order to reduce overfitting, neural networks are typically trained with data augmentation, the practice of artificially generating additional training data via label-preserving transformations of existing training examples. While these types of transformations make intuitive sense, recent work has demonstrated that even non-label-preserving data augmentation can be surprisingly effective, examining this type of data augmentation through linear combinations of pairs of examples. Despite their effectiveness, little is known about why such methods work. In this work, we aim to explore a new, m…
RWTH Aachen University（亚琛工业大学）；NVIDIA（英伟达）；ETH AI Center（苏黎世联邦理工学院AI中心）
We present Mix3D, a data augmentation technique for segmenting large-scale 3D scenes. Since scene context helps reasoning about object semantics, current works focus on models with large capacity and receptive fields that can fully capture the global context of an input 3D scene. However, strong contextual priors can have detrimental implications like mistaking a pedestrian crossing the street for a car. In this work, we focus on the importance of balancing global scene context and local geometry, with the goal of generalizing beyond the contextual priors in the training set. In particular, we…
Data augmentation is an essential technique for improving recognition accuracy in object recognition using deep learning. Methods that generate mixed data from multiple data sets, such as mixup, can acquire new diversity that is not included in the training data, and thus contribute significantly to accuracy improvement. However, since the data selected for mixing are randomly sampled throughout the training process, there are cases where appropriate classes or data are not selected. In this study, we propose a data augmentation method that calculates the distance between classes based on clas…
We describe Countersynth, a conditional generative model of diffeomorphic deformations that induce label-driven, biologically plausible changes in volumetric brain images. The model is intended to synthesise counterfactual training data augmentations for downstream discriminative modelling tasks where fidelity is limited by data imbalance, distributional instability, confounding, or underspecification, and exhibits inequitable performance across distinct subpopulations. Focusing on demographic attributes, we evaluate the quality of synthesized counterfactuals with voxel-based morphometry, clas…
GenMix: Combining Generative and Mixture Data Augmentation for Medical Image Classification
School of Electrical Engineering（电气工程学院）；Korea Advanced Institute of Science and Technology（韩国科学技术院）；Department of Software Convergence（软件融合系）；Seoul Women’s University（首尔女子大学）
University of Colorado at Colorado Springs（科罗拉多大学科罗拉多斯普林斯分校）
Facial attributes are emerging soft biometrics that have the potential to reject non-matches, for example, based on mismatching gender. To be usable in stand-alone systems, facial attributes must be extracted from images automatically and reliably. In this paper, we propose a simple yet effective solution for automatic facial attribute extraction by training a deep convolutional neural network (DCNN) for each facial attribute separately, without using any pre-training or dataset augmentation, and we obtain new state-of-the-art facial attribute classification results on the CelebA benchmark. To…
University of Amsterdam（阿姆斯特丹大学）；Google Research, Amsterdam（谷歌研究院阿姆斯特丹分部）；Peking University（北京大学）
This paper introduces data augmentation for point clouds by interpolation between examples. Data augmentation by interpolation has shown to be a simple and effective approach in the image domain. Such a mixup is however not directly transferable to point clouds, as we do not have a one-to-one correspondence between the points of two different objects. In this paper, we define data augmentation between point clouds as a shortest path linear interpolation. To that end, we introduce PointMixup, an interpolation method that generates new examples through an optimal assignment of the path function …
Xidian University（西安电子科技大学）；Intellifusion（云天励飞）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））
With the wide and deep adoption of deep learning models in real applications, there is an increasing need to model and learn the representations of the neural networks themselves. These models can be used to estimate attributes of different neural network architectures such as the accuracy and latency, without running the actual training or inference tasks. In this paper, we propose a neural architecture representation model that can be used to estimate these attributes holistically. Specifically, we first propose a simple and effective tokenizer to encode both the operation and topology infor…
