MorphAtt：一种用于脉冲视觉Transformer中高效多头注意力处理的神经形态加速器
MorphAtt: A Neuromorphic Accelerator for Efficient Multi-Head Attention Processing in Spiking Vision Transformers
- New York University（纽约大学）
- University of Tehran（德黑兰大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
针对脉冲视觉Transformer推理能效低的问题，提出神经形态加速器MorphAtt，通过级联硬件模块优化多头自注意力处理，在32nm工艺下实现20.3-29.1 TOPS/W能效，优于现有技术。
AI中文摘要：
脉冲视觉Transformer（SViT）被开发为传统视觉Transformer（ViT）在边缘计算机视觉任务中的节能替代方案。然而，庞大的参数数量和复杂的多头自注意力（MHSA）操作使得在SViT推理中实现高能效变得具有挑战性，尤其是在资源严格受限的应用中。为了最大化SViT处理的效率增益，我们提出了MorphAtt，一种新颖的数字加速器，通过精简处理流程加速SViT推理。具体而言，它使用级联硬件模块处理MHSA操作：脉冲查询-键-值生成器（SpikeQKV）、低复杂度脉冲多头自注意力引擎（SpikeAtten）以及重参数化卷积（RepConv）模块。为了缓解片上存储器访问和数据重用中的流量拥塞，数据流中集成了专门的模块间缓冲区。在采用32nm CMOS技术综合下，MorphAtt实现了792-1605 GOPS的吞吐量，功耗约为39-55 mW，面积为1.5 mm^2，能效达到20.3-29.1 TOPS/W。这些结果还表明，我们的MorphAtt比现有最先进技术提供了更好的性能和效率权衡，从而在边缘实现高能效的基于视觉的AI系统。
英文摘要：
Spiking Vision Transformers (SViTs) are developed as an energy-efficient alternative to conventional ViTs for computer vision tasks at the edge. However, huge parameter counts and complex multi-head self-attention (MHSA) operations make it challenging to achieve high energy efficiency in SViT inference, especially in tightly constrained applications. To maximize efficiency gains of SViT processing, we propose MorphAtt, a novel digital accelerator that expedites SViT inference through streamlined processing. Specifically, it processes MHSA operations using cascaded hardware modules: a Spiking Query-Key-Value generator (SpikeQKV), a low-complexity Spiking Multi-Head Self-Attention engine (SpikeAtten), and Reparameterization Convolution (RepConv) modules. To mitigate traffic congestion in on-chip memory accesses and data reuse, specialized inter-module buffers are integrated within the dataflow. Under synthesis using 32nm CMOS technology, MorphAtt achieves 792-1605 GOPS of throughput, while incurring ~39-55 mW of power consumption and 1.5 mm^2 of area, which lead to 20.3-29.1 TOPS/W of energy efficiency. These results also demonstrate that our MorphAtt offers better performance and efficiency trade-offs than state-of-the-art, thereby enabling highly energy-efficient vision-based AI systems at the edge.
