从四分之一到全部：通过浮点指数重映射与参数共享加速大语言模型投机解码
From Quarter to All: Accelerating Speculative LLM Decoding via Floating-Point Exponent Remapping and Parameter Sharing
AI总结：
针对大语言模型推理延迟高的问题，提出算法硬件协同设计的SPEQ投机解码方法，复用全精度模型权重比特构建量化草稿模型，搭配可重构阵列高效执行，在15个模型任务上较多种基线实现最高2.07倍无损加速。
摘要：
大语言模型在各类任务上表现出色，但因参数量庞大，推理延迟很高。量化虽能缩减模型体积，但与全精度模型相比通常会造成性能下降。投机解码虽能保持无损，但一般会带来额外开销。我们提出SPEQ——一种算法与硬件协同设计的投机解码方法，它利用全精度模型权重的部分比特构成量化草稿模型，从而免去额外的训练或存储开销。可重构处理单元阵列可高效执行草稿生成与验证两个阶段。在15个大语言模型及任务上的实验结果表明，SPEQ相较于FP16、Olive和Tender，分别实现了2.07倍、1.53倍和1.45倍的加速比。
英文摘要：
Large language models achieve impressive performance across diverse tasks but exhibit high inference latency due to their large parameter sizes. While quantization reduces model size, it often leads to performance degradation compared to the full model. Speculative decoding remains lossless but typically incurs extra overheads. We propose SPEQ, an algorithm-hardware co-designed speculative decoding method that uses part of the full-model weight bits to form a quantized draft model, thereby eliminating additional training or storage overhead. A reconfigurable processing element array enables efficient execution of both the draft and verification passes. Experimental results across 15 LLMs and tasks demonstrate that SPEQ achieves speedups of 2.07x, 1.53x, and 1.45x compared over FP16, Olive, and Tender, respectively.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
