arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1907.08985cs.DC
收藏

实时的DNN推理的多FPGA超线性加速

Achieving Super-Linear Speedup across Multi-FPGA for Real-Time DNN Inference

Weiwen Jiang, Edwin H. -M. Sha, Xinyi Zhang, Lei Yang, Qingfeng Zhuge, Yiyu Shi, Jingtong Hu

摘要:

具有低延迟要求的实时的【形】深度神经网络(DNN)推理对于云计算(例如,苹果的Siri)和边缘计算(例如,Google/Waymo的无人驾驶汽车)。基于FPGA的DNN加速器已经证明了卓越的灵活性和性能;此外,对于低批量的实时的【形】推理,FPGA有望实现进一步的性能提升。然而,从单FPGA设计的性能增益是由有限的片上资源的阻碍。在本文中,我们采用多个FPGA协同运行DNN的目标是实现超线性加速对单FPGA设计。在实现这样的系统时,我们发现了两个阻碍我们实现设计目标的障碍:(1)每个DNN层缺乏明确的分区方案以充分利用并行性,以及(2)由于DNN的大小不断增长,片外存储器和加速器之间的带宽不足。为了解决这些问题,我们提出了一个通用的框架,“超级LIP”,它可以支持不同类型的DNN。本文以卷积神经网络(CNN)为载体,对Super-LIP进行了研究。我们首先制定一个准确的系统级模型,以支持最佳分区方案的探索。然后,我们开发了一种新的设计方法,以有效地减轻沉重的负担,内存带宽的流量从内存总线到FPGA间的链接。我们在ZCU 102 FPGA板上实现了Super-LIP。结果表明,与最先进的单FPGA设计相比,使用2个FPGA的Super-LIP可以实现3.48倍的加速比。更重要的是,随着FPGA数量的增加,系统延迟可以进一步减少,同时保持高能效。

英文摘要:

Real-time Deep Neural Network (DNN) inference with low-latency requirement has become increasingly important for numerous applications in both cloud computing (e.g., Apple's Siri) and edge computing (e.g., Google/Waymo's driverless car). FPGA-based DNN accelerators have demonstrated both superior flexibility and performance; in addition, for real-time inference with low batch size, FPGA is expected to achieve further performance improvement. However, the performance gain from the single-FPGA design is obstructed by the limited on-chip resource. In this paper, we employ multiple FPGAs to cooperatively run DNNs with the objective of achieving super-linear speed-up against single-FPGA design. In implementing such systems, we found two barriers that hinder us from achieving the design goal: (1) the lack of a clear partition scheme for each DNN layer to fully exploit parallelism, and (2) the insufficient bandwidth between the off-chip memory and the accelerator due to the growing size of DNNs. To tackle these issues, we propose a general framework, "Super-LIP", which can support different kinds of DNNs. In this paper, we take Convolutional Neural Network (CNN) as a vehicle to illustrate Super-LIP. We first formulate an accurate system-level model to support the exploration of best partition schemes. Then, we develop a novel design methodology to effectively alleviate the heavy loads on memory bandwidth by moving traffic from memory bus to inter-FPGA links. We implement Super-LIP based on ZCU102 FPGA boards. Results demonstrate that Super-LIP with 2 FPGAs can achieve 3.48x speedup, compared to the state-of-the-art single-FPGA design. What is more, as the number of FPGAs scales up, the system latency can be further reduced while maintaining high energy efficiency.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑