CAT:基于Versal ACAP的定制化Transformer加速器框架
CAT: Customized Transformer Accelerator Framework on Versal ACAP
AI总结:
针对GPU硬件定制有限、FPGA设计难度高的问题,本文提出基于Versal ACAP的CAT定制化Transformer加速器框架,通过软硬件协同约束定制属性生成加速器,在VCK5000上实现了优于GPU、传统FPGA及现有SOTA的吞吐量与能效表现。
摘要:
Transformer最初以GPU作为设计平台,但GPU仅能实现有限的硬件定制。尽管FPGA具备较强的定制能力,但其设计解空间庞大,设计难度较高。Versal ACAP是一种以AI Engine为核心的异构计算架构,在硬件定制方面远比GPU灵活,且相比传统FPGA拥有更优、更小的设计解空间。因此,本文提出了定制化Transformer加速器框架(Customized Transformer Accelerator Framework,CAT),通过该框架可在Versal ACAP上衍生出定制化的Transformer加速器系列。CAT框架具备抽象的加速器架构设计思路,将Transformer解构并高效映射到硬件中,其中包含多种可定制属性。通过CAT框架的定制与优化策略,底层硬件与上层模型对这些可定制属性共同约束并决策,最终形成定制化加速器。我们采用7nm AMD Versal ACAP VCK5000开发板,基于CAT框架实现了针对不同Transformer模型的加速器。实验表明,与8nm Nvidia GPU A10G、16nm AMD FPGA ZCU102以及7nm AMD Versal ACAP VC190(当前最优水平SOTA)相比,我们分别实现了2.41倍、49.50倍和1.32倍的最高吞吐量提升,最高能效提升分别为7.80倍、6.19倍和1.15倍。
英文摘要:
Transformer uses GPU as the initial design platform, but GPU can only perform limited hardware customization. Although FPGA has strong customization ability, the design solution space is huge and the design difficulty is high. Versal ACAP is a heterogeneous computing architecture with AI Engine as the core. It is far more flexible than GPU in hardware customization, and has better and smaller design solution space than traditional FPGA. Therefore, this paper proposes the Customized Transformer Accelerator Framework(CAT), through the CAT framework, a customized Transformer accelerator family can be derived on Versal ACAP, CAT framework has an abstract accelerator architecture design idea, which deconstructs and efficiently maps the Transformer into the hardware, which contains a variety of customizable properties. Through the customization and optimization strategy of the CAT framework, the underlying hardware and the upper model jointly constrain and decide on these customizable properties, and finally form a customized accelerator. We use a 7 nm AMD Versal ACAP VCK5000 development board to implement accelerators for different Transformer models based on the CAT framework. Experiments show that we achieve the highest throughput gains of 2.41x, 49.50x, and 1.32x compared to 8 nm Nvidia GPU A10G, 16 nm AMD FPGA ZCU102, and 7 nm AMD Versal ACAP VC190(SOTA). The highest energy efficiency gains are 7.80x, 6.19x and 1.15x, respectively.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。