ITA：一种面向量化Transformer的能效注意力与Softmax加速器
ITA: An Energy-Efficient Attention and Softmax Accelerator for Quantized Transformers
AI总结：
本文提出ITA加速器，通过8位量化和仅基于整数的流式softmax实现，在嵌入式系统上实现高效Transformer推理，达到16.9 TOPS/W能效和5.93 TOPS/mm²面积效率。
摘要：
Transformer网络已成为自然语言处理任务的最先进方法，并在计算机视觉和音频处理等其他领域日益普及。然而，由于其高算术强度、大内存需求和复杂的数据流依赖性，Transformer模型的高效硬件加速面临新挑战。本文提出ITA，一种面向Transformer及相关模型的新型加速器架构，通过利用8位量化和仅在整数值上运行的创新性softmax实现，旨在嵌入式系统上实现高效推理。通过在流式模式下进行即时计算，我们的softmax实现最大程度减少了数据移动和能耗。ITA实现了16.9 TOPS/W的能效，可与最先进的Transformer加速器相媲美，同时在22纳米全耗尽绝缘体上硅技术、0.8V电压下，以5.93 TOPS/mm²的面积效率超越它们。
英文摘要：
Transformer networks have emerged as the state-of-the-art approach for natural language processing tasks and are gaining popularity in other domains such as computer vision and audio processing. However, the efficient hardware acceleration of transformer models poses new challenges due to their high arithmetic intensities, large memory requirements, and complex dataflow dependencies. In this work, we propose ITA, a novel accelerator architecture for transformers and related models that targets efficient inference on embedded systems by exploiting 8-bit quantization and an innovative softmax implementation that operates exclusively on integer values. By computing on-the-fly in streaming mode, our softmax implementation minimizes data movement and energy consumption. ITA achieves competitive energy efficiency with respect to state-of-the-art transformer accelerators with 16.9 TOPS/W, while outperforming them in area efficiency with 5.93 TOPS/mm$^2$ in 22 nm fully-depleted silicon-on-insulator technology at 0.8 V.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
