arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2605.09375cs.AR
收藏

31.1 一种14.08至135.69token/s的基于逻辑的堆叠异常自由大语言模型加速器,具备块聚类权重压缩和自适应并行推测解码

31.1 A 14.08-to-135.69Token/s ReRAM-on-Logic Stacked Outlier-Free Large-Language-Model Accelerator with Block-Clustered Weight-Compression and Adaptive Parallel-Speculative-Decoding

Hong Kong University of Science and Technology(香港科技大学);AI Chip Center for Emerging Smart System(新兴智能系统芯片中心);Hefei Reliance Memory(合肥芯联集成);Zhejiang University(浙江大学)
Pingcheng Dong, Yonghao Tan, Xuejiao Liu, Peng Luo, Yu Liu, Di Pang, Songchen Ma, Xijie Huang, Shih-Yang Liu, Dong Zhang, Zhichao Lu, Luhong Liang, Chi-Ying Tsu… 展开作者

Pingcheng Dong, Yonghao Tan, Xuejiao Liu, Peng Luo, Yu Liu, Di Pang, Songchen Ma, Xijie Huang, Shih-Yang Liu, Dong Zhang, Zhichao Lu, Luhong Liang, Chi-Ying Tsui, Fengbin Tu, Liang Zhao, Kwang-Ting Cheng

AI总结:

本文提出了一种基于55nm推测解码的LLM加速器,采用基于突起的面对面ReRAM-on-logic堆叠技术,通过局部旋转单元实现无异常低比特量化,结合块级向量量化减少权重EMA开销,并采用自适应并行推测解码方案提升资源和带宽利用率。

摘要:

本文提出了一种基于55nm推测解码的LLM加速器,采用基于突起的面对面ReRAM-on-logic堆叠技术。其特点包括局部旋转单元用于实现无异常低比特量化,堆叠感知的PNM架构与块级向量量化协同设计以减少权重EMA开销,以及自适应并行推测解码方案,配备无序调度器以提高资源和带宽利用率。我们的芯片在token/s方面达到14.08至135.69,相比普通推测解码方案实现了4.46至7.17倍的加速效果。

英文摘要:

This work presents a 55nm speculative decoding-based LLM accelerator with bumping-based face-to-face ReRAM-on-logic stacking technology. It features a local rotation unit for outlier-free low-bit quantization, a stacking-aware PNM architecture co-designed with blockwise vector quantization to reduce weight EMA overheads, and an adaptive parallel speculative decoding scheme with an out-of-order scheduler for high resource and bandwidth utilization. Our chip achieves 14.08-to-135.69token/s and 4.46-to-7.17x speedup over vanilla speculative decoding.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑