arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.04940cs.ARcs.LG
收藏

一种基于电荷型存内计算的Transformer模数混合注意力加速器

An Analog and Digital Hybrid Attention Accelerator for Transformers with Charge-based In-memory Computing

University of California, San Diego(加利福尼亚大学圣迭戈分校)

Ashkan Moradifirouzabadi, Divya Sri Dodla, Mingu Kang

AI总结:

针对Transformer自注意力计算复杂度高、内存访问频繁的问题,提出一款65nm工艺的模数混合注意力加速器,通过模拟存内计算核心实时剪枝75%低得分令牌、数字核心处理剩余令牌保证精度,实现了高能效与高面积效率。

摘要:

注意力机制是Transformer的核心计算内核,用于计算整个输入序列的两两相关性。自注意力计算中的计算复杂度与频繁内存访问给系统带来了巨大负担,尤其是在序列长度增加时。本文提出了一款采用65nm CMOS工艺的模数混合处理器,用于加速Transformer的注意力机制。我们设计了一个模拟存内计算(CIM)核心,可在运行时以超低功耗和延迟平均剪枝约75%的低得分令牌。此外,数字处理器仅对模拟CIM核心选出的约25%未剪枝令牌执行精确计算,避免精度下降。实测结果显示,模拟核心与片上系统(SoC)的峰值能效分别为14.8和1.65 TOPS/W,峰值面积效率分别为976.6和79.4 GOPS/mm²。

英文摘要:

The attention mechanism is a key computing kernel of Transformers, calculating pairwise correlations across the entire input sequence. The computing complexity and frequent memory access in computing self-attention put a huge burden on the system especially when the sequence length increases. This paper presents an analog and digital hybrid processor to accelerate the attention mechanism for transformers in 65nm CMOS technology. We propose an analog computing-in-memory (CIM) core, which prunes ~75% of low-score tokens on average during runtime at ultra-low power and delay. Additionally, a digital processor performs precise computations only for ~25% unpruned tokens selected by the analog CIM core, preventing accuracy degradation. Measured results show peak energy efficiency of 14.8 and 1.65 TOPS/W, and peak area efficiency of 976.6 and 79.4 GOPS/mm$^\mathrm{2}$ in the analog core and the system-on-chip (SoC), respectively.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑