arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1901.08263cs.NEcs.LG
收藏

QGAN:量化的生成对抗网络

QGAN: Quantized Generative Adversarial Networks

Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系);Department of Electrical and Computer Engineering, University of California, Santa Barbara(加利福尼亚大学圣巴巴拉分校电气与计算机工程系)

Peiqi Wang, Dongsheng Wang, Yu Ji, Xinfeng Xie, Haoxuan Song, XuXin Liu, Yongqiang Lyu, Yuan Xie

摘要:

生成对抗神经网络(GANs)的密集计算和内存需求阻碍了其在智能手机等边缘设备上的实际部署。尽管在CNN的模型降阶方面取得了成功,但神经网络量化方法尚未在GANs上进行研究,主要面临量化算法的有效性和训练GAN模型的不稳定性两方面的问题。在本文中,我们首先对将现有的成功方法应用于并行GANs进行了广泛的研究。我们的观察表明,它们都没有生成具有合理质量的样本,因为模型权重中的量化值的代表性不足,并且生成器和神经网络对量化方法表现出不同的敏感性。受这些观察的启发,我们开发了一种新的基于EM算法的GANs量化方法,称为QGAN。我们还提出了一个多精度算法,以帮助找到量化的GAN模型的最佳比特数与相应的结果质量。在CIFAR-10和CelebA上的实验表明,QGAN可以将GANs扩展到甚至1位或2位表示,其质量与原始模型相当。

英文摘要:

The intensive computation and memory requirements of generative adversarial neural networks (GANs) hinder its real-world deployment on edge devices such as smartphones. Despite the success in model reduction of CNNs, neural network quantization methods have not yet been studied on GANs, which are mainly faced with the issues of both the effectiveness of quantization algorithms and the instability of training GAN models. In this paper, we start with an extensive study on applying existing successful methods to quantize GANs. Our observation reveals that none of them generates samples with reasonable quality because of the underrepresentation of quantized values in model weights, and the generator and discriminator networks show different sensitivities upon quantization methods. Motivated by these observations, we develop a novel quantization method for GANs based on EM algorithms, named as QGAN. We also propose a multi-precision algorithm to help find the optimal number of bits of quantized GAN models in conjunction with corresponding result qualities. Experiments on CIFAR-10 and CelebA show that QGAN can quantize GANs to even 1-bit or 2-bit representations with results of quality comparable to original models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑