Generative adversarial networks (GANs) are successfully used for image synthesis but are known to face instability during training. In contrast, probabilistic diffusion models (DMs) are stable and generate high-quality images, at the cost of an expensive sampling procedure. In this paper, we introduce a simple method to allow GANs to stably converge to their theoretical optimum, while bringing in the denoising machinery from DMs. These models are combined into a simpler model (ATME) that only requires a forward pass during inference, making predictions cheaper and more accurate than DMs and popular GANs. ATME breaks an information asymmetry existing in most GAN models in which the discriminator has spatial knowledge of where the generator is failing. To restore the information symmetry, the generator is endowed with knowledge of the entropic state of the discriminator, which is leveraged to allow the adversarial game to converge towards equilibrium. We demonstrate the power of our method in several image-to-image translation tasks, showing superior performance than state-of-the-art methods at a lesser cost. Code is available at https://github.com/DLR-MI/atme
Combating Mode Collapse in GANs via Manifold Entropy Estimation
King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学（KAUST））；Jarvis Lab, Tencent（腾讯Jarvis实验室）；YouTu Lab, Tencent（腾讯优图实验室）；Shenzhen University（深圳大学）
NAAMII（尼泊尔人工智能与信息研究所）；Imperial College London（伦敦帝国理工学院）；KAIST（韩国科学技术院）
Multi-domain image-to-image translation with conditional Generative Adversarial Networks (GANs) can generate highly photo realistic images with desired target classes, yet these synthetic images have not always been helpful to improve downstream supervised tasks such as image classification. Improving downstream tasks with synthetic examples requires generating images with high fidelity to the unknown conditional distribution of the target class, which many labeled conditional GANs attempt to achieve by adding soft-max cross-entropy loss based auxiliary classifier in the discriminator. As rece…
ST Electronics - SUTD Cyber Security Laboratory（新加坡科技电子-新加坡科技设计大学网络安全实验室）；Singapore University of Technology and Design（新加坡科技设计大学）
We propose to improve unconditional Generative Adversarial Networks (GAN) by training the self-supervised learning with the adversarial process. In particular, we apply self-supervised learning via the geometric transformation on input images and assign the pseudo-labels to these transformed images. (i) In addition to the GAN task, which distinguishes data (real) versus generated (fake) samples, we train the discriminator to predict the correct pseudo-labels of real transformed samples (classification task). Importantly, we find out that simultaneously training the discriminator to classify th…
City University of Hong Kong（香港城市大学）；The Education University of Hong Kong（香港教育大学）；Northwestern Polytechnical University（西北工业大学）；CodeHatch Corp.（CodeHatch公司）
Unsupervised learning with generative adversarial networks (GANs) has proven hugely successful. Regular GANs hypothesize the discriminator as a classifier with the sigmoid cross entropy loss function. However, we found that this loss function may lead to the vanishing gradients problem during the learning process. To overcome such a problem, we propose in this paper the Least Squares Generative Adversarial Networks (LSGANs) which adopt the least squares loss function for the discriminator. We show that minimizing the objective function of LSGAN yields minimizing the Pearson $χ^2$ divergence. T…
Accurately matching visual and textual data in cross-modal retrieval has been widely studied in the multimedia community. To address these challenges posited by the heterogeneity gap and the semantic gap, we propose integrating Shannon information theory and adversarial learning. In terms of the heterogeneity gap, we integrate modality classification and information entropy maximization adversarially. For this purpose, a modality classifier (as a discriminator) is built to distinguish the text and image modalities according to their different statistical properties. This discriminator uses its…
Tooploox（Tooploox公司）；Voyage（Voyage公司）；Warsaw University of Technology（华沙理工大学）
In this paper, we propose a novel regularization method for Generative Adversarial Networks, which allows the model to learn discriminative yet compact binary representations of image patches (image descriptors). We employ the dimensionality reduction that takes place in the intermediate layers of the discriminator network and train binarized low-dimensional representation of the penultimate layer to mimic the distribution of the higher-dimensional preceding layers. To achieve this, we introduce two loss terms that aim at: (i) reducing the correlation between the dimensions of the binarized lo…
Universitat Politecnica de Catalunya（加泰罗尼亚理工大学）；Facebook AML；Insight Center for Data Analytics（Insight数据分析中心）；Dublin City University（都柏林城市大学）；Barcelona Supercomputing Center（巴塞罗那超级计算中心）
We introduce SalGAN, a deep convolutional neural network for visual saliency prediction trained with adversarial examples. The first stage of the network consists of a generator model whose weights are learned by back-propagation computed from a binary cross entropy (BCE) loss over downsampled versions of the saliency maps. The resulting prediction is processed by a discriminator network trained to solve a binary classification task between the saliency maps generated by the generative stage and the ground truth ones. Our experiments show how adversarial training allows reaching state-of-the-a…
We introduce a challenging training scheme of conditional GANs, called open-set semi-supervised image generation, where the training dataset consists of two parts: (i) labeled data and (ii) unlabeled data with samples belonging to one of the labeled data classes, namely, a closed-set, and samples not belonging to any of the labeled data classes, namely, an open-set. Unlike the existing semi-supervised image generation task, where unlabeled data only contain closed-set samples, our task is more general and lowers the data collection cost in practice by allowing open-set samples to appear. Thank…
University of California, Merced（加州大学默塞德分校）；NEC Laboratories America（NEC美国实验室）；National Taiwan University（国立台湾大学）；Academia Sinica, Taiwan（台湾中央研究院）；Google Cloud（谷歌云）
We propose a method for semi-supervised semantic segmentation using an adversarial network. While most existing discriminators are trained to classify input images as real or fake on the image level, we design a discriminator in a fully convolutional manner to differentiate the predicted probability maps from the ground truth segmentation distribution with the consideration of the spatial resolution. We show that the proposed discriminator can be used to improve semantic segmentation accuracy by coupling the adversarial loss with the standard cross entropy loss of the proposed model. In additi…
In this paper, we propose a novel generative model named Stacked Generative Adversarial Networks (SGAN), which is trained to invert the hierarchical representations of a bottom-up discriminative network. Our model consists of a top-down stack of GANs, each learned to generate lower-level representations conditioned on higher-level representations. A representation discriminator is introduced at each feature hierarchy to encourage the representation manifold of the generator to align with that of the bottom-up discriminative network, leveraging the powerful discriminative representations to gui…
City University of Hong Kong（香港城市大学）；The Education University of Hong Kong（香港教育大学）；Northwestern Polytechnical University（西北工业大学）；CodeHatch Corp.（CodeHatch公司）
Unsupervised learning with generative adversarial networks (GANs) has proven to be hugely successful. Regular GANs hypothesize the discriminator as a classifier with the sigmoid cross entropy loss function. However, we found that this loss function may lead to the vanishing gradients problem during the learning process. To overcome such a problem, we propose in this paper the Least Squares Generative Adversarial Networks (LSGANs) which adopt the least squares loss for both the discriminator and the generator. We show that minimizing the objective function of LSGAN yields minimizing the Pearson…
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Shanghai Jiao Tong University（上海交通大学）
Conditional generative models aim to learn the underlying joint distribution of data and labels to achieve conditional data generation. Among them, the auxiliary classifier generative adversarial network (AC-GAN) has been widely used, but suffers from the problem of low intra-class diversity of the generated samples. The fundamental reason pointed out in this paper is that the classifier of AC-GAN is generator-agnostic, which therefore cannot provide informative guidance for the generator to approach the joint distribution, resulting in a minimization of the conditional entropy that decreases …
Techno India University, West Bengal（西孟加拉邦Techno印度大学）
In this paper we present a novel method for an automated diagnosis of breast carcinoma through multilevel iterative variational mode decomposition (VMD) and textural features encompassing Zernaike moments, fractal dimension and entropy features namely, Kapoor entropy, Renyi entropy, Yager entropy features are extracted from VMD components. The proposed method considers the histopathological image as a set of multidimensional spatially-evolving signals. ReliefF algorithm is used to select the discriminatory features and statistically most significant features are fed to squares support vector m…
Hefei University of Technology（合肥工业大学）；Dalian University of Technology（大连理工大学）
Person re-identification (re-ID) is a task of matching pedestrians under disjoint camera views. To recognise paired snapshots, it has to cope with large cross-view variations caused by the camera view shift. Supervised deep neural networks are effective in producing a set of non-linear projections that can transform cross-view images into a common feature space. However, they typically impose a symmetric architecture, yielding the network ill-conditioned on its optimisation. In this paper, we learn view-invariant subspace for person re-ID, and its corresponding similarity metric using an adver…
Automated medical image segmentation is an important step in many medical procedures. Recently, deep learning networks have been widely used for various medical image segmentation tasks, with U-Net and generative adversarial nets (GANs) being some of the commonly used ones. Foreground-background class imbalance is a common occurrence in medical images, and U-Net has difficulty in handling class imbalance because of its cross entropy (CE) objective function. Similarly, GAN also suffers from class imbalance because the discriminator looks at the entire image to classify it as real or fake. Since…
MDA GAN: Adversarial-Learning-based 3-D Seismic Data Interpolation and Reconstruction for Complex Missing
College of computer science and technology, China University of Petroleum (East China)（中国石油大学（华东）计算机科学与技术学院）；College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）；Shengli Oilfield Company, SINOPEC（中国石化胜利油田分公司）；Center on Frontiers of Computing Studies, Peking University（北京大学计算前沿研究中心）
本文提出三维 MDA GAN 框架，通过三判别器、特征拼接和 TCE 损失解决地震数据复杂缺失重建问题，在高比例随机缺失、连续缺失及富断层盐体场景中优于现有方法。
Feature Engineering is Not Dead: Reviving Classical Machine Learning with Entropy, HOG, and LBP Feature Fusion for Image Classification
Department of Physics and Engineering Physics, Tulane University（物理与工程物理系，路易斯安那大学）；Institute of Applied Geosciences, Karlsruhe Institute of Technology (KIT)（应用地球科学研究所，卡尔斯鲁厄技术大学）；Cure and Care Wellness Pvt Ltd（治愈与关怀健康私人有限公司）；Amritha Vidya Vishwapeetham（阿米特哈大学）
