基于FPGA的高性能CNN加速的低精度浮点算法
Low Precision Floating-point Arithmetic for High Performance FPGA-based CNN Acceleration
摘要:
低精度数据表示对于减少卷积神经网络(CNN)的存储大小和内存访问非常重要。然而,现有的方法有两个主要的限制:(1)需要重新训练以保持深度CNN的准确性,以及(2)需要16位浮点或8位定点以获得良好的准确性。在本文中,我们提出了一种低精度(8位)浮点(LPFP)量化方法,基于FPGA的加速,以克服上述限制。在没有任何重新训练的情况下,LPFP找到了一个最佳的8位数据表示,其前1/前5精度损失可以忽略不计(在我们的实验中分别在0.5%/0.3%以内,并且明显优于现有的深度CNN方法)。此外,我们通过一个4位乘加器(MAC)和一个3位加法器实现了一个8位LPFP乘法,从而在一个DSP芯片上实现了4个8位LPFP乘法,而一个DSP只能实现两个8位定点乘法。在六个典型的CNN上进行的推理实验表明,平均而言,我们的吞吐量比Intel i9 CPU提高了64.5倍,比现有的FPGA加速器提高了1.5倍。特别是对于VGG 16和YOLO,与最近的六个FPGA加速器相比,我们将平均吞吐量分别提高了3.5倍和27.5倍,每个DSP的平均吞吐量分别提高了4.1倍和5倍。据我们所知,这是第一次深入研究将CNN推理的一次乘法简化为一个4位MAC,并在一个DSP中实现四次乘法,同时保持相当的准确性,无需任何重新训练。
英文摘要:
Low precision data representation is important to reduce storage size and memory access for convolutional neural networks (CNNs). Yet, existing methods have two major limitations: (1) requiring re-training to maintain accuracy for deep CNNs, and (2) needing 16-bit floating-point or 8-bit fixed-point for a good accuracy. In this paper, we propose a low precision (8-bit) floating-point (LPFP) quantization method for FPGA-based acceleration to overcome the above limitations. Without any re-training, LPFP finds an optimal 8-bit data representation with negligible top-1/top-5 accuracy loss (within 0.5%/0.3% in our experiments, respectively, and significantly better than existing methods for deep CNNs). Furthermore, we implement one 8-bit LPFP multiplication by one 4-bit multiply-adder (MAC) and one 3-bit adder, and therefore implement four 8-bit LPFP multiplications using one DSP slice of Xilinx Kintex 7 family (KC705 in this paper) while one DSP can implement only two 8-bit fixed-point multiplications. Experiments on six typical CNNs for inference show that on average, we improve throughput by 64.5x over Intel i9 CPU and by 1.5x over existing FPGA accelerators. Particularly for VGG16 and YOLO, compared to six recent FPGA accelerators, we improve average throughput by 3.5x and 27.5x and improve average throughput per DSP by 4.1x and 5x, respectively. To the best of our knowledge, this is the first in-depth study to simplify one multiplication for CNN inference to one 4-bit MAC and implement four multiplications within one DSP while maintaining comparable accuracy without any re-training.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。