FusionCIM: 通过融合驱动的计算存内架构加速大语言模型推理
FusionCIM: Accelerating LLM Inference with Fusion-Driven Computing-in-Memory Architecture
AI总结：
本文提出FusionCIM架构，通过融合驱动的计算存内架构提升LLM推理效率，核心方法包括混合CIM流水线、QO静态数据流和模式感知在线softmax机制，实现3.86倍能效提升和1.98倍加速。
摘要：
本文提出FusionCIM，一种由操作融合驱动的计算存内（CIM）加速器架构，用于高效可扩展的大语言模型（LLM）推理，具有三个关键创新：（1）混合CIM流水线架构，将QKT计算映射到基于内积的CIM（IP-CIM）和基于外积的CIM（OP-CIM）上的点积计算融合；（2）QO静态数据流消除CIM中的重复KV加载和转置融合下的K矩阵访问，显著提高芯片上的数据重用率；（3）模式感知的在线softmax机制利用注意力分数的分布规律，减少非线性融合的指数重新缩放开销。实验结果表明，FusionCIM在LLaMA-3模型上实现了相比先前SOTA CIM设计的3.86倍能效提升和1.98倍加速，系统层面的能效达到29.4 TOPS/W。
英文摘要：
In this paper, we propose FusionCIM, an operator-fusion-driven compute-in-memory (CIM) accelerator architecture for efficient and scalable LLM inference, with three key innovations: (1) a hybrid CIM pipeline architecture that maps QKT computation on inner-product-based CIM (IP-CIM) and PV aggregation on outer-product-based CIM (OP-CIM) for efficient matrix multiplications fusion; (2) a QO-stationary dataflow that eliminates repeated KV loading in CIM and K-matrix access in buffer under transpose fusion, significantly improving data reuse on chip; and (3) a pattern-aware online-softmax mechanism that exploits distribution regularities of attention scores to reduce exponential rescaling overhead for non-linear fusion. Experimental results on LLaMA-3 model show that FusionCIM achieves up to 3.86x energy saving, and 1.98x speedup compared with prior SOTA CIM-based designs with 29.4 TOPS/W energy efficiency at the system level.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
