PIM-GPT:一种面向自回归Transformer的混合存内计算加速器
PIM-GPT: A Hybrid Process-in-Memory Accelerator for Autoregressive Transformers
AI总结:
针对GPT类自回归Transformer推理的内存受限瓶颈,提出基于DRAM存内计算与ASIC协同的PIM-GPT加速器,通过软硬件协同优化实现了远超CPU、GPU的推理速度与能效。
摘要:
仅解码器架构的Transformer模型(如GPT)通过自回归预测下一个token,在文本生成任务中展现出卓越性能。然而,当前硬件系统运行GPT的效率受限于低计算内存比和高内存访问开销。存内计算(Process-in-Memory, PIM)架构可最大限度减少片外数据移动并利用高内部带宽,是加速GPT推理这类内存受限任务的理想候选方案。\n本研究提出一种PIM加速器PIM-GPT,可实现GPT推理的端到端加速,兼具高性能与高能效。PIM-GPT采用基于DRAM的PIM设计,直接在DRAM芯片内执行乘累加(MAC)运算,无需将矩阵数据移出芯片;非线性函数运算与数据通信则由专用集成电路(ASIC)支持。在软件层面,研究设计了映射方案,通过在DRAM通道与bank间拼接、划分矩阵,最大化数据局部性与计算并行度,以充分利用所有可用的存内计算单元。研究通过电路综合与事件驱动的时钟周期精确模拟器验证了PIM-GPT架构的效率。总体而言,在参数规模最高达14亿的8个GPT模型上,PIM-GPT相较于GPU和CPU基线分别实现了41-137倍、631-1074倍的速度提升,以及123-383倍、320-602倍的能效提升。
英文摘要:
Decoder-only Transformer models such as GPT have demonstrated exceptional performance in text generation, by autoregressively predicting the next token. However, the efficacy of running GPT on current hardware systems is bounded by low compute-to-memory-ratio and high memory access. Process-in-memory (PIM) architectures can minimize off-chip data movement and utilize high internal bandwidth. They stand out as promising candidates for accelerating memory-bounded tasks such as GPT inference. In this work, we propose a PIM accelerator, PIM-GPT, which achieves end-to-end acceleration of GPT inference with high performance and high energy efficiency. PIM-GPT leverages DRAM-based PIM designs for executing multiply-accumulate (MAC) operations directly in the DRAM chips, eliminating the need to move matrix data off-chip. Non-linear functions and data communication is supported by an application specific integrated chip (ASIC). At the software level, mapping schemes are designed to maximize data locality and computation parallelism by concatenating and partitioning matrices among DRAM channels and banks to utilize all available in-memory computation units. The efficiency of the PIM-GPT architecture is verified through circuit synthesis and an event-driven clock-cycle accurate simulator. Overall, PIM-GPT achieves 41$-$137$\times$, 631$-$1074$\times$ speedup and 123$-$383$\times$, 320$-$602$\times$ energy efficiency over GPU and CPU baseline on 8 GPT models with up to 1.4 billion parameters.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。