一种面向边缘生成式AI的灵活模板，配备高精度加速Softmax与GELU
A Flexible Template for Edge Generative AI with High-Accuracy Accelerated Softmax & GELU
AI总结：
本研究针对边缘Transformer生成式AI中非线性运算的性能瓶颈，提出含新型SoftEx加速器的异构紧耦合集群加速模板，在精度损失极小的情况下显著提升了softmax、GELU运算及端到端ViT推理的性能与能效。
摘要：
基于Transformer的生成式人工智能（GenAI）模型在自然语言处理、计算机视觉和音频处理等广泛领域取得了显著成果。然而，这是以复杂度提升以及需要softmax和GELU等复杂非线性运算为代价的。尽管Transformer的计算主要由矩阵乘法（MatMul）主导，但这些非线性运算可能会成为性能瓶颈，尤其是在使用专用硬件加速MatMul算子的情况下。\n在本研究中，我们提出了一种GenAI BFloat16 Transformer加速模板，该模板基于一个异构紧耦合集群，包含256KiB共享SRAM、8个通用RISC-V核心、一个24x8脉动阵列MatMul加速器，以及一款用于Transformer softmax和GELU非线性运算的新型加速器SoftEx。SoftEx引入了一种近似指数算法，在效率（相比glibc实现实现121倍加速）与精度（平均相对误差0.14%）之间取得平衡。\n在12nm工艺下，SoftEx的面积为0.039 mm²，仅占集群的3.22%，集群的工作频率为1.12 GHz。与在RISC-V核心上运行的优化软件相比，SoftEx实现了显著提升，将softmax和GELU的计算速度分别最高提升10.8倍和5.11倍，同时将其能耗分别最高降低10.8倍和5.29倍。这些改进使得端到端ViT推理工作负载的吞吐量提升1.58倍（0.8V下为310 GOPS），能效提升1.42倍（0.55V下为1.34 TOPS/W）。
英文摘要：
Transformer-based generative Artificial Intelligence (GenAI) models achieve remarkable results in a wide range of fields, including natural language processing, computer vision, and audio processing. However, this comes at the cost of increased complexity and the need of sophisticated non-linearities such as softmax and GELU. Even if Transformers are computationally dominated by matrix multiplications (MatMul), these non-linearities can become a performance bottleneck, especially if dedicated hardware is used to accelerate MatMul operators. In this work, we introduce a GenAI BFloat16 Transformer acceleration template based on a heterogeneous tightly-coupled cluster containing 256KiB of shared SRAM, 8 general-purpose RISC-V cores, a 24x8 systolic array MatMul accelerator, and a novel accelerator for Transformer softmax and GELU non-linearities: SoftEx. SoftEx introduces an approximate exponentiation algorithm balancing efficiency (121x speedup over glibc's implementation) with accuracy (mean relative error of 0.14%). In 12nm technology, SoftEx occupies 0.039 mm$^2$, only 3.22% of the cluster, which achieves an operating frequency of 1.12 GHz. Compared to optimized software running on the RISC-V cores, SoftEx achieves significant improvements, accelerating softmax and GELU computations by up to 10.8x and 5.11x, respectively, while reducing their energy consumption by up to 10.8x and 5.29x. These enhancements translate into a 1.58x increase in throughput (310 GOPS at 0.8V) and a 1.42x improvement in energy efficiency (1.34 TOPS/W at 0.55V) on end-to-end ViT inference workloads.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
