arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.14854cs.CVcs.AIeess.IV
收藏

PEANO-ViT:视觉Transformer中非线性操作的能效近似方法

PEANO-ViT: Power-Efficient Approximations of Non-Linearities in Vision Transformers

University of Southern California(南加州大学)

Mohammad Erfan Sadeghi, Arash Fayyazi, Seyedarmin Azizi, Massoud Pedram

AI总结:

针对视觉Transformer在FPGA上部署时非线性函数功耗高的问题,提出PEANO-ViT框架,通过无除法层归一化、多尺度softmax近似和分段线性GELU近似,在精度损失极小的情况下大幅提升三类非线性操作的能效,助力ViT在资源功耗受限FPGA上高效部署。

摘要:

视觉Transformer(ViT)在硬件平台,尤其是现场可编程门阵列(FPGA)上的部署面临诸多挑战,主要源于其非线性函数(特别是层归一化、softmax和高斯误差线性单元(GELU))的巨大计算与功耗需求。这些关键函数因复杂的数学运算,以及FPGA固有的资源数量和架构限制,给高效硬件实现带来了显著障碍。PEANO-ViT提出了一种新颖的层归一化层实现优化方法,引入无除法技术同时近似除法与平方根函数。此外,PEANO-ViT采用多尺度除法策略消除softmax层中的除法运算,并辅以基于Pade的指数函数近似方法。最后,PEANO-ViT为GELU函数引入了分段线性近似,经过精心设计以规避GELU相关的高计算量操作。在我们的综合评估中,PEANO-ViT的精度损失极小(DeiT-B上≤0.5%),同时显著提升了能效,层归一化、softmax和GELU的能效分别提升了1.91倍、1.39倍和8.01倍。这一提升是通过大幅减少这些非线性运算的DSP、LUT和寄存器数量实现的。因此,PEANO-ViT使得视觉Transformer能够在资源和功耗受限的FPGA平台上高效部署。

英文摘要:

The deployment of Vision Transformers (ViTs) on hardware platforms, specially Field-Programmable Gate Arrays (FPGAs), presents many challenges, which are mainly due to the substantial computational and power requirements of their non-linear functions, notably layer normalization, softmax, and Gaussian Error Linear Unit (GELU). These critical functions pose significant obstacles to efficient hardware implementation due to their complex mathematical operations and the inherent resource count and architectural limitations of FPGAs. PEANO-ViT offers a novel approach to streamlining the implementation of the layer normalization layer by introducing a division-free technique that simultaneously approximates the division and square root function. Additionally, PEANO-ViT provides a multi-scale division strategy to eliminate division operations in the softmax layer, aided by a Pade-based approximation for the exponential function. Finally, PEANO-ViT introduces a piece-wise linear approximation for the GELU function, carefully designed to bypass the computationally intensive operations associated with GELU. In our comprehensive evaluations, PEANO-ViT exhibits minimal accuracy degradation (<= 0.5% for DeiT-B) while significantly enhancing power efficiency, achieving improvements of 1.91x, 1.39x, 8.01x for layer normalization, softmax, and GELU, respectively. This improvement is achieved through substantial reductions in DSP, LUT, and register counts for these non-linear operations. Consequently, PEANO-ViT enables efficient deployment of Vision Transformers on resource- and power-constrained FPGA platforms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑