arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.05499cs.AI
收藏

SDFP:基于FIT剪枝模型的投机解码用于无训练和即插即用的大语言模型加速

SDFP: Speculative Decoding with FIT-Pruned Models for Training-Free and Plug-and-Play LLM Acceleration

Tsinghua University(清华大学);Advanced Micro Devices, Inc.(先进微器件公司)

Hanyu Wei, Zunhai Su, Peng Lu, Chao Li, Spandan Tiwari, Ashish Sirasao, Yuhan Dong

AI总结:

SDFP通过FIT剪枝技术实现无训练、即插即用的大语言模型加速,提升解码速度而不影响输出分布。

摘要:

大型语言模型(LLMs)支撑着交互式多媒体应用,如描述生成、检索、推荐和创意内容生成,但其自回归解码会带来显著的延迟。投机解码通过轻量级草稿模型减少延迟,但部署通常受限于获取、调优和维护有效草稿模型的成本和复杂性。最近的方法通常需要辅助训练或专门化,甚至无训练方法也会产生昂贵的搜索或优化。我们提出了SDFP,一个完全无训练且即插即用的框架,通过Fisher信息迹(FIT)基于的层剪枝来构建给定LLM的草稿模型。利用层敏感性作为输出扰动的代理,SDFP移除低影响层以获得紧凑的草稿,同时保持与原始模型的兼容性以进行标准的投机验证。SDFP不需要额外的训练、超参数调优或单独维护的草稿,使快速、部署友好的草稿构建成为可能。在基准测试中,SDFP在不改变目标模型输出分布的情况下,提供1.32倍至1.5倍的解码加速,支持低延迟的多媒体应用。

英文摘要:

Large language models (LLMs) underpin interactive multimedia applications such as captioning, retrieval, recommendation, and creative content generation, yet their autoregressive decoding incurs substantial latency. Speculative decoding reduces latency using a lightweight draft model, but deployment is often limited by the cost and complexity of acquiring, tuning, and maintaining an effective draft model. Recent approaches usually require auxiliary training or specialization, and even training-free methods incur costly search or optimization. We propose SDFP, a fully training-free and plug-and-play framework that builds the draft model via Fisher Information Trace (FIT)-based layer pruning of a given LLM. Using layer sensitivity as a proxy for output perturbation, SDFP removes low-impact layers to obtain a compact draft while preserving compatibility with the original model for standard speculative verification. SDFP needs no additional training, hyperparameter tuning, or separately maintained drafts, enabling rapid, deployment-friendly draft construction. Across benchmarks, SDFP delivers 1.32x-1.5x decoding speedup without altering the target model's output distribution, supporting low-latency multimedia applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑