arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.01762cs.AIcs.CLcs.LG
收藏

PRISM: 参数化重构推理用于推测采样草案模型

PRISM: Parametrically Refactoring Inference for Speculative Sampling Draft Models

Xuliang Wang, Yuetao Chen, Maochan Zhen, Fang Liu, Xinzhou Zheng, Xingwu Liu, Hong Xu, Ming Li

AI总结:

PRISM通过参数化重构推理方法,有效解耦模型容量与推理成本,提升LLM解码效率。

摘要:

大型语言模型(LLMs)由于其自回归性质而面临解码缓慢的问题。推测解码方法已逐渐成为加速LLM解码的有希望的解决方案,引起了系统和人工智能研究社区的关注。最近,追求更好的草案质量推动了一种趋势,即使用参数更大的草案模型,这不可避免地引入了显著的计算开销。尽管现有工作试图在预测准确性与计算延迟之间取得平衡,但我们通过架构创新解决了这一根本矛盾。我们提出了PRISM,将每个预测步骤的计算分解到不同的参数集上,重构草案模型的计算路径,成功地将模型容量与推理成本解耦。通过广泛的实验,我们证明PRISM优于所有现有的草案架构,在保持最小的草案延迟的同时实现了极长的接受长度,从而获得卓越的端到端加速效果。我们还重新审视了缩放定律,发现PRISM在数据量扩大时比其他草案架构更具扩展性。通过严格且公平的比较,我们展示了PRISM能够将一个已经高度优化的推理引擎的解码吞吐量提高超过2.6倍。

英文摘要:

Large Language Models (LLMs), constrained by their auto-regressive nature, suffer from slow decoding. Speculative decoding methods have emerged as a promising solution to accelerate LLM decoding, attracting attention from both systems and AI research communities. Recently, the pursuit of better draft quality has driven a trend toward parametrically larger draft models, which inevitably introduces substantial computational overhead. While existing work attempts to balance the trade-off between prediction accuracy and compute latency, we address this fundamental dilemma through architectural innovation. We propose PRISM, which disaggregates the computation of each predictive step across different parameter sets, refactoring the computational pathways of draft models to successfully decouple model capacity from inference cost. Through extensive experiments, we demonstrate that PRISM outperforms all existing draft architectures, achieving exceptional acceptance lengths while maintaining minimal draft latency for superior end-to-end speedup. We also re-examine scaling laws with PRISM, revealing that PRISM scales more effectively with expanding data volumes than other draft architectures. Through rigorous and fair comparison, we show that PRISM boosts the decoding throughput of an already highly optimized inference engine by more than 2.6x.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑