面向可持续FPGA视觉Transformer推理的近似注意力加权方法
Approximate Attention Weighting for Sustainable FPGA-Based Vision Transformer Inference
AI总结:
提出一种无BRAM的近似注意力加权单元,用16段分段线性函数近似softmax中的自然指数,在Zynq-7020上实现,精度损失仅0.20%。
摘要:
视觉Transformer通过自注意力捕获图像区域的全局上下文,重塑了计算机视觉。这使得它们适用于边缘视觉检测和监控,如可再生能源基础设施、工业质量控制、医学成像和自主系统传感。然而,在小型FPGA上部署ViT仍然具有挑战性,因为自注意力中的softmax阶段需要指数评估和归一化,这在硬件中成本高昂。现有实现通常依赖CORDIC流水线或基于BRAM的查找表,增加了面积和功耗。本文提出了一种无BRAM的近似注意力加权单元,用于基于FPGA的ViT推理。所提出的设计使用完全由分布式LUT结构实现的16段分段线性函数来近似softmax中的自然指数。与基2近似不同,自然指数公式保留了预训练的注意力温度,避免了特定模型的重新校准。在Xilinx Zynq-7020上实现,完整的注意力行核心使用1444个LUT、77个DSP,且无BRAM,硬件精确仿真显示,在ViT系列模型上,与精确softmax参考相比,绝对top-1精度差异在0.20%以内。这些结果证明了所提出的核心在资源受限的边缘AI平台上实现节能ViT推理的潜力。
英文摘要:
Vision Transformers have reshaped computer vision by using self-attention to capture global context across image regions. This makes them attractive for edge visual inspection and monitoring in applications such as renewable-energy infrastructure, industrial quality control, medical imaging, and autonomous-system sensing. However, deploying ViTs on small FPGAs remains challenging because the softmax stage in self-attention requires exponential evaluation and normalization, which are costly in hardware. Existing implementations often rely on CORDIC pipelines or BRAM-based look-up tables, increasing area and power consumption. This paper presents a BRAM-free approximate attention-weighting unit for FPGA-based ViT inference. The proposed design approximates the natural exponential in softmax using a 16-segment piecewise-linear function implemented entirely with distributed LUT fabric. Unlike base-2 approximations, the natural-exponential formulation preserves the pre-trained attention temperature and avoids model-specific recalibration. Implemented on a Xilinx Zynq-7020, the complete attention-row core uses 1444 LUTs, 77 DSPs, and no BRAM, while hardware-accurate emulation shows accuracy within a \(0.20\%\) absolute top-1 difference from the exact-softmax reference on ViT-family models. These results demonstrate the potential of the proposed core for energy-efficient ViT inference on resource-constrained edge-AI platforms.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。