HG-PIPE：利用混合粒度流水线加速 Vision Transformer
HG-PIPE: Vision Transformer Acceleration with Hybrid-Grained Pipeline
AI总结：
本文提出 HG-PIPE 流水线 FPGA 加速器，通过混合粒度架构、数据流与并行耦合及 LUT 近似消除气泡并降低资源开销，显著提升 ViT 吞吐与能效。
摘要：
利用现场可编程门阵列（FPGA）加速 Vision Transformer（ViT）具有良好前景，但也充满挑战。现有基于 FPGA 的 ViT 加速器主要依赖时序架构，通过复用相同硬件块处理不同算子，因而承受大量内存访问开销。粗粒度或细粒度的流水线架构会在空间上展开 ViT 计算，以提高内存访问效率。然而，它们通常受到严重的硬件资源约束，以及 ViT 全局计算依赖所引发的流水线气泡影响。本文提出 HG-PIPE，这是一种用于高吞吐量、低延迟 ViT 处理的流水线 FPGA 加速器。HG-PIPE 采用混合粒度流水线架构以降低片上缓冲区成本，并将计算数据流与并行设计耦合，以消除流水线气泡。HG-PIPE 还引入精心设计的近似方法，利用充足的查找表（LUT）实现线性和非线性算子，从而缓解资源约束。在 ZCU102 FPGA 上，与 AutoViTAcc 等现有最优加速器相比，HG-PIPE 的吞吐量提升 2.78 倍，资源效率提升 2.52 倍。借助 VCK190 FPGA，HG-PIPE 在单设备上实现端到端 ViT 加速，达到 7118 images/s，比 V100 GPU 快 2.81 倍。
英文摘要：
Vision Transformer (ViT) acceleration with field programmable gate array (FPGA) is promising but challenging. Existing FPGA-based ViT accelerators mainly rely on temporal architectures, which process different operators by reusing the same hardware blocks and suffer from extensive memory access overhead. Pipelined architectures, either coarse-grained or fine-grained, unroll the ViT computation spatially for memory access efficiency. However, they usually suffer from significant hardware resource constraints and pipeline bubbles induced by the global computation dependency of ViT. In this paper, we introduce HG-PIPE, a pipelined FPGA accelerator for high-throughput and low-latency ViT processing. HG-PIPE features a hybrid-grained pipeline architecture to reduce on-chip buffer cost and couples the computation dataflow and parallelism design to eliminate the pipeline bubbles. HG-PIPE further introduces careful approximations to implement both linear and non-linear operators with abundant Lookup Tables (LUTs), thus alleviating resource constraints. On a ZCU102 FPGA, HG-PIPE achieves 2.78 times better throughput and 2.52 times better resource efficiency than the prior-art accelerators, e.g., AutoViTAcc. With a VCK190 FPGA, HG-PIPE realizes end-to-end ViT acceleration on a single device and achieves 7118 images/s, which is 2.81 times faster than a V100 GPU.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
