AccUDNN: A GPU Memory Efficient Accelerator for Training Ultra-deep Neural Networks
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
Jinrong Guo, Wantao Liu, Wang Wang, Qu Lu, Songlin Hu, Jizhong Han, Ruixuan Li
Typically, Ultra-deep neural network(UDNN) tends to yield high-quality model, but its training process is usually resource intensive and time-consuming. Modern GPU's scarce DRAM capacity is the primary bottleneck that hinders the trainability and the training efficiency of UDNN. In this paper, we present "AccUDNN", an accelerator that aims to make the utmost use of finite GPU memory resources to speed up the training process of UDNN. AccUDNN mainly includes two modules: memory optimizer and hyperparameter tuner. Memory optimizer develops a performance-model guided dynamic swap out/in strategy, by offloading appropriate data to host memory, GPU memory footprint can be significantly slashed to overcome the restriction of trainability of UDNN. After applying the memory optimization strategy, hyperparameter tuner is designed to explore the efficiency-optimal minibatch size and the matched learning rate. Evaluations demonstrate that AccUDNN cuts down the GPU memory requirement of ResNet-152 from more than 24GB to 8GB. In turn, given 12GB GPU memory budget, the efficiency-optimal minibatch size can reach 4.2x larger than original Caffe. Benefiting from better utilization of single GPU's computing resources and fewer parameter synchronization of large minibatch size, 7.7x speed-up is achieved by 8 GPUs' cluster without any communication optimization and no accuracy losses.
A GPU-Outperforming FPGA Accelerator Architecture for Binary Convolutional Neural Networks
Arizona State University(亚利桑那州立大学);Nanyang Technological University(南洋理工大学)
基于FPGA的卷积神经网络(CNN)硬件加速器由于其比GPU更高的能量效率而受到广泛关注。然而,基于FPGA的解决方案要实现比GPU更高的吞吐量是具有挑战性的。在本文中,我们证明了当CNN在权重和激活的二进制约束下进行训练时,FPGA加速在吞吐量和能效方面都是一种优越的解决方案。具体来说,我们提出了一种优化的FPGA加速器架构,专为逐位卷积和归一化而设计,具有大规模的空间并行性和深流水线阶段。FPGA加速器的一个关键优势是其性能对数据批量大小不敏感,而GPU加速的性能在很大程度上取决于数据的批量大小。实验结果表明,在Virtex-7 FPGA上运行的二进制CNN的加速器架构比Titan X GPU快8.3倍,能效高75倍,可以处理小批量的在线个人请求。对于处理大批量静态数据,该解决方案在吞吐量方面与Titan X GPU不相上下,同时提供9.5倍的能效。
Energy-Efficient Inference Accelerator for Memory-Augmented Neural Networks on an FPGA
Seoul National University(首尔国立大学)
记忆增强神经网络(MANN)是为问答任务而设计的。很难在为其他神经网络(NN)设计的加速器上有效地运行MANN,特别是在移动设备上,因为MANN需要循环数据路径和与外部存储器访问相关的各种类型的操作。我们实现了一个基于数据流架构的现场可编程门阵列(FPGA)的人工神经网络加速器。推理时间也减少了推理阈值,这是一个基于数据的最大内积搜索专用于自然语言的任务。对bAbI数据的测量显示,加速器的能量效率(FLOPS/kJ)比NVIDIA TITAN V GPU高出约125倍,在推理阈值化的情况下增加到140倍