We aim to build image generation models that generalize to new domains from few examples. To this end, we first investigate the generalization properties of classic image generators, and discover that autoencoders generalize extremely well to new domains, even when trained on highly constrained data. We leverage this insight to produce a robust, unsupervised few-shot image generation algorithm, and introduce a novel training procedure based on recovering an image from data augmentations. Our Augmentation-Interpolative AutoEncoders synthesize realistic images of novel objects from only a few reference images, and outperform both prior interpolative models and supervised few-shot image generators. Our procedure is simple and lightweight, generalizes broadly, and requires no category labels or other supervision during training.
School of Computer Science, The University of Sydney（悉尼大学计算机科学学院）；School of Computer Science, University of Technology Sydney（悉尼科技大学计算机科学学院）
Class imbalance occurs in many real-world applications, including image classification, where the number of images in each class differs significantly. With imbalanced data, the generative adversarial networks (GANs) leans to majority class samples. The two recent methods, Balancing GAN (BAGAN) and improved BAGAN (BAGAN-GP), are proposed as an augmentation tool to handle this problem and restore the balance to the data. The former pre-trains the autoencoder weights in an unsupervised manner. However, it is unstable when the images from different categories have similar features. The latter is …
ETH Zurich（苏黎世联邦理工学院）；University of Zurich（苏黎世大学）；Stanford University（斯坦福大学）；Stanford Stroke Center（斯坦福中风中心）；Institute for Biomedical Engineering（生物医学工程研究所）
We show that high quality, diverse and realistic-looking diffusion-weighted magnetic resonance images can be synthesized using deep generative models. Based on professional neuroradiologists' evaluations and diverse metrics with respect to quality and diversity of the generated synthetic brain images, we present two networks, the Introspective Variational Autoencoder and the Style-Based GAN, that qualify for data augmentation in the medical field, where information is saved in a dispatched and inhomogeneous way and access to it is in many aspects restricted.
Image classification datasets are often imbalanced, characteristic that negatively affects the accuracy of deep-learning classifiers. In this work we propose balancing GAN (BAGAN) as an augmentation tool to restore balance in imbalanced datasets. This is challenging because the few minority-class images may not be enough to train a GAN. We overcome this issue by including during the adversarial training all available images of majority and minority classes. The generative model learns useful features from majority classes and uses these to generate images for minority classes. We apply class c…
Herein, we present a system for hyperspectral image segmentation that utilizes multiple class--based denoising autoencoders which are efficiently trained. Moreover, we present a novel hyperspectral data augmentation method for labelled HSI data using linear mixtures of pixels from each class, which helps the system with edge pixels which are almost always mixed pixels. Finally, we utilize a deep neural network and morphological hole-filling to provide robust image classification. Results run on the Salinas dataset verify the high performance of the proposed algorithm.
Lomonosov Moscow State University（罗蒙诺索夫莫斯科国立大学）；NRU Higher School of Economics（国立研究型大学高等经济学院）
Traffic sign recognition is a well-researched problem in computer vision. However, the state of the art methods works only for frequent sign classes, which are well represented in training datasets. We consider the task of rare traffic sign detection and classification. We aim to solve that problem by using synthetic training data. Such training data is obtained by embedding synthetic images of signs in the real photos. We propose three methods for making synthetic signs consistent with a scene in appearance. These methods are based on modern generative adversarial network (GAN) architectures.…
Yale University（耶鲁大学）；Massachusetts Institute of Technology（麻省理工学院）；Northwestern University（西北大学）
Aesthetics are critically important to market acceptance. In the automotive industry, an improved aesthetic design can boost sales by 30% or more. Firms invest heavily in designing and testing aesthetics. A single automotive "theme clinic" can cost over $100,000, and hundreds are conducted annually. We propose a model to augment the commonly-used aesthetic design process by predicting aesthetic scores and automatically generating innovative and appealing product designs. The model combines a probabilistic variational autoencoder (VAE) with adversarial components from generative adversarial net…
Mixup is a powerful data augmentation method that interpolates between two or more examples in the input or feature space and between the corresponding target labels. Many recent mixup methods focus on cutting and pasting two or more objects into one image, which is more about efficient processing than interpolation. However, how to best interpolate images is not well defined. In this sense, mixup has been connected to autoencoders, because often autoencoders "interpolate well", for instance generating an image that continuously deforms into another. In this work, we revisit mixup from the int…
Moscow Institute of Physics and Technology（莫斯科物理技术学院）；Lomonosov Moscow State University（莫斯科国立大学（罗蒙诺索夫莫斯科国立大学））
Finding an interpretable non-redundant representation of real-world data is one of the key problems in Machine Learning. Biological neural networks are known to solve this problem quite well in unsupervised manner, yet unsupervised artificial neural networks either struggle to do it or require fine tuning for each task individually. We associate this with the fact that a biological brain learns in the context of the relationships between observations, while an artificial network does not. We also notice that, though a naive data augmentation technique can be very useful for supervised learning…
Institute of Digital Technologies, Loughborough University（拉夫堡大学数字技术研究所）
Synthesizing a realistic image from textual description is a major challenge in computer vision. Current text to image synthesis approaches falls short of producing a highresolution image that represent a text descriptor. Most existing studies rely either on Generative Adversarial Networks (GANs) or Variational Auto Encoders (VAEs). GANs has the capability to produce sharper images but lacks the diversity of outputs, whereas VAEs are good at producing a diverse range of outputs, but the images generated are often blurred. Taking into account the relative advantages of both GANs and VAEs, we pr…
SOS: A Shuffle Order Strategy for Data Augmentation in Industrial Human Activity Recognition
Faculty of Computer Science（计算机科学系）；Engineering Ho Chi Minh city University of Technology- Vietnam National University Ho Chi Minh City, Vietnam（工程学院-胡志明市技术大学-越南国家大学胡志明市）；Department of Biomedical Engineering Ho Chi Minh city University of Technology- Vietnam National University Ho Chi Minh City, Vietnam（生物医学工程系-胡志明市技术大学-越南国家大学胡志明市）
Chongqing University（重庆大学）；University of Delaware（特拉华大学）
Data augmentation is a powerful technique for improving the performance of the few-shot classification task. It generates more samples as supplements, and then this task can be transformed into a common supervised learning issue for solution. However, most mainstream data augmentation based approaches only consider the single modality information, which leads to the low diversity and quality of generated features. In this paper, we present a novel multi-modal data augmentation approach named Dizygotic Conditional Variational AutoEncoder (DCVAE) for addressing the aforementioned issue. DCVAE co…
We propose a new strategy to improve the accuracy and robustness of image classification. First, we train a baseline CNN model. Then, we identify challenging regions in the feature space by identifying all misclassified samples, and correctly classified samples with low confidence values. These samples are then used to train a Variational AutoEncoder (VAE). Next, the VAE is used to generate synthetic images. Finally, the generated synthetic images are used in conjunction with the original labeled images to train a new model in a semi-supervised fashion. Empirical results on benchmark datasets …
Self-supervised learning (SSL) enables learning useful inductive biases through utilizing pretext tasks that require no labels. The unlabeled nature of SSL makes it especially important for whole slide histopathological images (WSIs), where patch-level human annotation is difficult. Masked Autoencoders (MAE) is a recent SSL method suitable for digital pathology as it does not require negative sampling and requires little to no data augmentations. However, the domain shift between natural images and digital pathology images requires further research in designing MAE for patch-level WSIs. In thi…
IBM（国际商业机器公司）；Yonsei University（延世大学）；Oak Ridge National Laboratory（橡树岭国家实验室）
Knowing the similarity between sets of data has a number of positive implications in training an effective model, such as assisting an informed selection out of known datasets favorable to model transfer or data augmentation problems with an unknown dataset. Common practices to estimate the similarity between data include comparing in the original sample space, comparing in the embedding space from a model performing a certain task, or fine-tuning a pretrained model with different datasets and evaluating the performance changes therefrom. However, these practices would suffer from shallow comp…
Pontifical Catholic University of Rio de Janeiro（里约热内卢天主教大学）；PETROBRAS（巴西国家石油公司）
Nowadays, subsurface salt body localization and delineation, also called semantic segmentation of salt bodies, are among the most challenging geophysicist tasks. Thus, identifying large salt bodies is notoriously tricky and is crucial for identifying hydrocarbon reservoirs and drill path planning. This work proposes a Data Augmentation method based on training two generative models to augment the number of samples in a seismic image dataset for the semantic segmentation of salt bodies. Our method uses deep learning models to generate pairs of seismic image patches and their respective salt mas…
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
We propose a composable framework for latent space image augmentation that allows for easy combination of multiple augmentations. Image augmentation has been shown to be an effective technique for improving the performance of a wide variety of image classification and generation tasks. Our framework is based on the Variational Autoencoder architecture and uses a novel approach for augmentation via linear transformation within the latent space itself. We explore losses and augmentation latent geometry to enforce the transformations to be composable and involuntary, thus allowing the transformat…
Deakin University（迪肯大学）；Islamic Azad University（伊斯兰阿扎德大学）；Isfahan University of Technology（伊斯法罕理工大学）；University of Birjand（比尔詹德大学）；Ferdowsi University of Mashhad（马什哈德菲尔多西大学）；K. N. Toosi University of Technology（K. N. 图西理工大学）；Universidad de Granada（格拉纳达大学）；University of Cambridge（剑桥大学）；Dibrugarh University（迪布鲁格尔大学）；Iran University of Medical Sciences（伊朗医科大学）；University of Bojnord（博杰纳德大学）；The George Institute for Global Health（乔治全球健康研究所）；University of Sydney（悉尼大学）
COVID-19 has caused many deaths worldwide. The automation of the diagnosis of this virus is highly desired. Convolutional neural networks (CNNs) have shown outstanding classification performance on image datasets. To date, it appears that COVID computer-aided diagnosis systems based on CNNs and clinical information have not yet been analysed or explored. We propose a novel method, named the CNN-AE, to predict the survival chance of COVID-19 patients using a CNN trained with clinical information. Notably, the required resources to prepare CT images are expensive and limited compared to those re…
Contrastive Tuning: A Little Help to Make Masked Autoencoders Forget
Faculty of Computer Science, University of Vienna, Vienna, Austria（维也纳大学计算机科学学院）；UniVie Doctoral School Computer Science, University of Vienna（维也纳大学计算机科学博士学院）；Institute of Advanced Research in Artificial Intelligence (IARAI)（高级人工智能研究所）
提出 MAE-CT 方法，结合 MIM 的效率与 ID 的对象聚焦能力，通过 NNCLR 目标在预训练 MAE 顶层诱导语义聚类，无需标签且计算开销低，在 ViT 模型上多项指标达 SOTA。
Multi-modal data generation with a deep metric variational autoencoder
Technical University of Denmark（丹麦技术大学）；Interacoustics Research Unit（国际听力研究单元）；Interacoustics A/S（国际听力公司）；Kamide ENT clinic（上田耳鼻咽喉科诊所）；Diatec Japan（迪亚泰克日本公司）
Universidade da Coruña（科鲁尼亚大学）；Universidad de Oviedo（奥维耶多大学）；CESGA（CESGA（无常见中文名，按原文））；NVIDIA Corporation（英伟达公司（NVIDIA Corporation））
Recommender systems (RS) are increasingly present in our daily lives, especially since the advent of Big Data, which allows for storing all kinds of information about users' preferences. Personalized RS are successfully applied in platforms such as Netflix, Amazon or YouTube. However, they are missing in gastronomic platforms such as TripAdvisor, where moreover we can find millions of images tagged with users' tastes. This paper explores the potential of using those images as sources of information for modeling users' tastes and proposes an image-based classification system to obtain personali…
The Hong Kong University of Science and Technology（香港科技大学）
Photorealistic facial expression synthesis from single face image can be widely applied to face recognition, data augmentation for emotion recognition or entertainment. This problem is challenging, in part due to a paucity of labeled facial expression data, making it difficult for algorithms to disambiguate changes due to identity and changes due to expression. In this paper, we propose the conditional difference adversarial autoencoder, CDAAE, for facial expression synthesis. The CDAAE takes a facial image of a previously unseen person and generates an image of that human face with a target e…
Building scalable models to learn from diverse, multimodal data remains an open challenge. For vision-language data, the dominant approaches are based on contrastive learning objectives that train a separate encoder for each modality. While effective, contrastive learning approaches introduce sampling bias depending on the data augmentations used, which can degrade performance on downstream tasks. Moreover, these methods are limited to paired image-text data, and cannot leverage widely-available unpaired data. In this paper, we investigate whether a large multimodal model trained purely via ma…
Fordham University（福特汉姆大学）；QS Investors, LLC（QS投资有限责任公司）；Grossman Medical School（格罗斯曼医学院）
In-scanner motion degrades the quality of magnetic resonance imaging (MRI) thereby reducing its utility in the detection of clinically relevant abnormalities. We introduce a deep learning-based MRI artifact reduction model (DMAR) to localize and correct head motion artifacts in brain MRI scans. Our approach integrates the latest advances in object detection and noise reduction in Computer Vision. Specifically, DMAR employs a two-stage approach: in the first, degraded regions are detected using the Single Shot Multibox Detector (SSD), and in the second, the artifacts within the found regions ar…
