We consider the task of photo-realistic unconditional image generation (generate high quality, diverse samples that carry the same visual content as the image) on mobile platforms using Generative Adversarial Networks (GANs). In this paper, we propose a novel approach to trade-off image generation accuracy of a GAN for the energy consumed (compute) at run-time called Scale-Energy Tradeoff GAN (SETGAN). GANs usually take a long time to train and consume a huge memory hence making it difficult to run on edge devices. The key idea behind SETGAN for an image generation task is for a given input image, we train a GAN on a remote server and use the trained model on edge devices. We use SinGAN, a single image unconditional generative model, that contains a pyramid of fully convolutional GANs, each responsible for learning the patch distribution at a different scale of the image. During the training process, we determine the optimal number of scales for a given input image and the energy constraint from the target edge device. Results show that with SETGAN's unique client-server-based architecture, we were able to achieve a 56% gain in energy for a loss of 3% to 12% SSIM accuracy. Also, with the parallel multi-scale training, we obtain around 4x gain in training time on the server.
GAN Compression: Efficient Architectures for Interactive Conditional GANs
Carnegie Mellon University(卡内基梅隆大学);Massachusetts Institute of Technology(麻省理工学院);University of Toronto(多伦多大学)
条件生成对抗网络(cGAN)已经为许多视觉和图形应用提供了可控的图像合成。然而,最近的cGAN比现代识别CNN的计算密集度高1-2个数量级。例如,GauGAN每个映像消耗281 G MAC,而MobileNet-v3消耗0.44 G MAC,这使得交互式部署变得困难。在这项工作中,我们提出了一个通用的压缩框架,用于减少cGAN中生成器的推理时间和模型大小。由于GAN训练的困难和生成器架构的差异,直接应用现有的压缩方法会产生较差的性能。我们以两种方式应对这些挑战。首先,为了稳定GAN训练,我们将原始模型的多个中间表示的知识转移到其压缩模型中,并统一非配对和配对学习。其次,我们的方法不是重用现有的CNN设计,而是通过神经架构搜索找到有效的架构。为了加速搜索过程,我们通过权重共享将模型训练和搜索解耦。实验证明了我们的方法在不同的监督设置,网络架构和学习方法的有效性。在不损失图像质量的情况下,我们将CycleGAN的计算量减少了21倍,Pix 2 pix减少了12倍,MUNIT减少了29倍,GauGAN减少了9倍,为交互式图像合成铺平了道路。
Learning Efficient GANs for Image Translation via Differentiable Masks and co-Attention Distillation
Xiamen University(厦门大学);Tencent(腾讯);Pinterest;Inception Institute of Artificial Intelligence(起源人工智能研究院);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
Generative Adversarial Networks (GANs) in Networking: A Comprehensive Survey & Evaluation
University of Tehran(德黑兰大学);Shahid Beheshti University(沙希德·贝赫什提大学);Standard Research Institute(标准研究院);University of East London(东伦敦大学);Middlesex University(密德萨斯大学)
FloGAN: Scenario-Based Urban Mobility Flow Generation via Conditional GANs and Dynamic Region Decoupling
Future Cities Laboratory Global(未来城市实验室全球);Singapore-ETH Centre(新加坡-ETH中心);College of Computing and Data Science(计算与数据科学学院);Nanyang Technological University(南洋理工大学);Department of Civil Engineering and Engineering Mechanics(土木工程与工程力学系);Columbia University(哥伦比亚大学)
Distilling portable Generative Adversarial Networks for Image Translation
Peking University(北京大学);Huawei Noah’s Ark Lab(华为诺亚方舟实验室);Huawei Consumer Business Group(华为消费者业务集团);National Engineering Laboratory for Video Technology, Peking University(北京大学视频技术国家工程实验室);Peng Cheng Laboratory(鹏城实验室);The University of Sydney(悉尼大学)
TRAMBA: A Hybrid Transformer and Mamba Architecture for Practical Audio and Bone Conduction Speech Super Resolution and Enhancement on Mobile and Wearable Platforms