用于快速且高能效大语言模型的模拟存内计算注意力机制
Analog In-Memory Computing Attention Mechanism for Fast and Energy-Efficient Large Language Models
AI总结:
本文提出基于增益单元的模拟存内计算自注意力架构,通过定制初始化算法无需从头训练即可达到GPT-2级性能,并将注意力延迟和能耗较GPU分别降低最多两个和五个数量级。
摘要:
由自注意力驱动的Transformer网络是大语言模型的核心。在生成式Transformer中,自注意力使用缓存存储器保存token投影,避免在每个时间步重复计算。然而,存储在GPU中的投影必须在每个新的生成步骤加载到SRAM中,造成延迟和能耗瓶颈。我们提出一种定制的自注意力存内计算架构,该架构基于新兴的电荷型存储器——增益单元(gain cells),可以在序列生成过程中高效写入以存储新token,并支持自注意力所需的并行模拟点积计算。然而,模拟增益单元电路引入非理想性和约束,阻碍了预训练模型的直接映射。为解决这一问题,我们设计了一种初始化算法,在不从头训练的情况下实现与GPT-2相当的文本处理性能。与GPU相比,我们的架构分别将注意力延迟和能耗降低最多两个和五个数量级,标志着向超快速、低功耗生成式Transformer迈出了重要一步。
英文摘要:
Transformer networks, driven by self-attention, are central to Large Language Models. In generative Transformers, self-attention uses cache memory to store token projections, avoiding recomputation at each time step. However, GPU-stored projections must be loaded into SRAM for each new generation step, causing latency and energy bottlenecks. We present a custom self-attention in-memory computing architecture based on emerging charge-based memories called gain cells, which can be efficiently written to store new tokens during sequence generation and enable parallel analog dot-product computation required for self-attention. However, the analog gain cell circuits introduce non-idealities and constraints preventing the direct mapping of pre-trained models. To circumvent this problem, we design an initialization algorithm achieving text processing performance comparable to GPT-2 without training from scratch. Our architecture respectively reduces attention latency and energy consumption by up to two and five orders of magnitude compared to GPUs, marking a significant step toward ultra-fast, low-power generative Transformers.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。