arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2111.14484cs.ET
收藏

生成对抗网络在无源RRAM纵横阵列上的节能实现

Energy-Efficient Implementation of Generative Adversarial Networks on Passive RRAM Crossbar Arrays

Siddharth Satyam, Honey Nikam, Shubham Sahay

摘要:

生成算法(如GANs)正处于无监督学习和大规模人工数据生成领域的下一场革命的风口浪尖。然而,GAN中的判别式和生成式网络的对抗(竞争)共同训练使得它们的计算密集型,并阻碍了它们在资源受限的物联网边缘设备上的部署。此外,在训练期间,判别网络和生成网络之间的频繁数据传输显著降低了von-Neumann GAN加速器(例如基于GPU和FPGA的加速器)的效率。因此,迫切需要开发用于GANs的超紧凑且节能的硬件加速器。为此,在这项工作中,我们建议利用无源RRAM交叉阵列来执行全连接GAN的关键操作:(a)生成器网络的真正随机噪声生成,(b)在前向传递和反向传播期间具有前所未有的能量效率的向量乘矩阵乘法,以及(C)使用硬件友好的曼哈顿规则进行原位对抗训练。我们的广泛的分析,利用实验校准的现象学模型无源RRAM交叉阵列揭示了一个不可预见的权衡之间的准确性和能量消耗,而训练的GAN网络与不同的噪声输入的发生器。此外,我们的研究结果表明,空间和时间的变化和真正的随机噪声,这是不可取的存储器应用,提高能量效率的GAN实现无源RRAM交叉阵列,而不会降低其准确性。

英文摘要:

Generative algorithms such as GANs are at the cusp of next revolution in the field of unsupervised learning and large-scale artificial data generation. However, the adversarial (competitive) co-training of the discriminative and generative networks in GAN makes them computationally intensive and hinders their deployment on the resource-constrained IoT edge devices. Moreover, the frequent data transfer between the discriminative and generative networks during training significantly degrades the efficacy of the von-Neumann GAN accelerators such as those based on GPU and FPGA. Therefore, there is an urgent need for development of ultra-compact and energy-efficient hardware accelerators for GANs. To this end, in this work, we propose to exploit the passive RRAM crossbar arrays for performing key operations of a fully-connected GAN: (a) true random noise generation for the generator network, (b) vector-by-matrix-multiplication with unprecedented energy-efficiency during the forward pass and backward propagation and (C) in-situ adversarial training using a hardware friendly Manhattan's rule. Our extensive analysis utilizing an experimentally calibrated phenomological model for passive RRAM crossbar array reveals an unforeseen trade-off between the accuracy and the energy dissipated while training the GAN network with different noise inputs to the generator. Furthermore, our results indicate that the spatial and temporal variations and true random noise, which are otherwise undesirable for memory application, boost the energy-efficiency of the GAN implementation on passive RRAM crossbar arrays without degrading its accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑