31.1 一种14.08至135.69token/s的基于逻辑的堆叠异常自由大语言模型加速器,具备块聚类权重压缩和自适应并行推测解码
31.1 A 14.08-to-135.69Token/s ReRAM-on-Logic Stacked Outlier-Free Large-Language-Model Accelerator with Block-Clustered Weight-Compression and Adaptive Parallel-Speculative-Decoding
AI总结:
本文提出了一种基于55nm推测解码的LLM加速器,采用基于突起的面对面ReRAM-on-logic堆叠技术,通过局部旋转单元实现无异常低比特量化,结合块级向量量化减少权重EMA开销,并采用自适应并行推测解码方案提升资源和带宽利用率。
摘要:
本文提出了一种基于55nm推测解码的LLM加速器,采用基于突起的面对面ReRAM-on-logic堆叠技术。其特点包括局部旋转单元用于实现无异常低比特量化,堆叠感知的PNM架构与块级向量量化协同设计以减少权重EMA开销,以及自适应并行推测解码方案,配备无序调度器以提高资源和带宽利用率。我们的芯片在token/s方面达到14.08至135.69,相比普通推测解码方案实现了4.46至7.17倍的加速效果。
英文摘要:
This work presents a 55nm speculative decoding-based LLM accelerator with bumping-based face-to-face ReRAM-on-logic stacking technology. It features a local rotation unit for outlier-free low-bit quantization, a stacking-aware PNM architecture co-designed with blockwise vector quantization to reduce weight EMA overheads, and an adaptive parallel speculative decoding scheme with an out-of-order scheduler for high resource and bandwidth utilization. Our chip achieves 14.08-to-135.69token/s and 4.46-to-7.17x speedup over vanilla speculative decoding.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。