arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.19315cs.NEcs.AIcs.ARcs.ET
收藏

用于快速且高能效大语言模型的模拟存内计算注意力机制

Analog In-Memory Computing Attention Mechanism for Fast and Energy-Efficient Large Language Models

Forschungszentrum Jülich(于利希研究中心);RWTH Aachen(亚琛工业大学)

Nathan Leroux, Paul-Philipp Manea, Chirag Sudarshan, Jan Finkbeiner, Sebastian Siegel, John Paul Strachan, Emre Neftci

AI总结:

本文提出基于增益单元的模拟存内计算自注意力架构,通过定制初始化算法无需从头训练即可达到GPT-2级性能,并将注意力延迟和能耗较GPU分别降低最多两个和五个数量级。

摘要:

由自注意力驱动的Transformer网络是大语言模型的核心。在生成式Transformer中,自注意力使用缓存存储器保存token投影,避免在每个时间步重复计算。然而,存储在GPU中的投影必须在每个新的生成步骤加载到SRAM中,造成延迟和能耗瓶颈。我们提出一种定制的自注意力存内计算架构,该架构基于新兴的电荷型存储器——增益单元(gain cells),可以在序列生成过程中高效写入以存储新token,并支持自注意力所需的并行模拟点积计算。然而,模拟增益单元电路引入非理想性和约束,阻碍了预训练模型的直接映射。为解决这一问题,我们设计了一种初始化算法,在不从头训练的情况下实现与GPT-2相当的文本处理性能。与GPU相比,我们的架构分别将注意力延迟和能耗降低最多两个和五个数量级,标志着向超快速、低功耗生成式Transformer迈出了重要一步。

英文摘要:

Transformer networks, driven by self-attention, are central to Large Language Models. In generative Transformers, self-attention uses cache memory to store token projections, avoiding recomputation at each time step. However, GPU-stored projections must be loaded into SRAM for each new generation step, causing latency and energy bottlenecks. We present a custom self-attention in-memory computing architecture based on emerging charge-based memories called gain cells, which can be efficiently written to store new tokens during sequence generation and enable parallel analog dot-product computation required for self-attention. However, the analog gain cell circuits introduce non-idealities and constraints preventing the direct mapping of pre-trained models. To circumvent this problem, we design an initialization algorithm achieving text processing performance comparable to GPT-2 without training from scratch. Our architecture respectively reduces attention latency and energy consumption by up to two and five orders of magnitude compared to GPUs, marking a significant step toward ultra-fast, low-power generative Transformers.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑