arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.03295cs.CLcs.AIcs.CV
收藏

POP:仅预填充剪枝用于高效大模型推理

POP: Prefill-Only Pruning for Efficient Large Model Inference

Wuhan University(武汉大学);OPPO Research Institute(OPPO研究院)

Junhui He, Zhihui Fu, Jun Wang, Qingan Li

AI总结:

本文提出POP剪枝方法,通过区分预填充和解码阶段,仅剪枝深度层以提升预填充效率,保留完整模型用于解码,实现1.37倍的预填充延迟提升且性能损失小。

摘要:

大型语言模型(LLMs)和视觉-语言模型(VLMs)展现了显著能力,但其部署受限于高昂的计算成本。现有结构化剪枝方法虽然硬件高效,但常导致显著的精度下降。本文认为,这种失败源于缺乏阶段意识的剪枝方法,忽视了预填充和解码阶段之间的不对称作用。通过引入虚拟门机制,我们的重要性分析揭示,深层层对下个token预测(解码)至关重要,但对上下文编码(预填充)则几乎冗余。基于此洞察,我们提出预填充-only剪枝(POP),一种阶段意识的推理策略,在计算密集的预填充阶段安全地省略深层层,同时在敏感的解码阶段保留完整模型。为实现阶段转换,我们引入独立的键值(KV)投影以维持缓存完整性,并引入边界处理策略以确保首个生成token的准确性。在Llama-3.1、Qwen3-VL和Gemma-3等模型上进行的广泛实验表明,POP在预填充延迟上实现了高达1.37倍的加速,且性能损失极小,有效克服了现有结构化剪枝方法在精度-效率权衡上的限制。

英文摘要:

Large Language Models (LLMs) and Vision-Language Models (VLMs) have demonstrated remarkable capabilities. However, their deployment is hindered by significant computational costs. Existing structured pruning methods, while hardware-efficient, often suffer from significant accuracy degradation. In this paper, we argue that this failure stems from a stage-agnostic pruning approach that overlooks the asymmetric roles between the prefill and decode stages. By introducing a virtual gate mechanism, our importance analysis reveals that deep layers are critical for next-token prediction (decode) but largely redundant for context encoding (prefill). Leveraging this insight, we propose Prefill-Only Pruning (POP), a stage-aware inference strategy that safely omits deep layers during the computationally intensive prefill stage while retaining the full model for the sensitive decode stage. To enable the transition between stages, we introduce independent Key-Value (KV) projections to maintain cache integrity, and a boundary handling strategy to ensure the accuracy of the first generated token. Extensive experiments on Llama-3.1, Qwen3-VL, and Gemma-3 across diverse modalities demonstrate that POP achieves up to 1.37$\times$ speedup in prefill latency with minimal performance loss, effectively overcoming the accuracy-efficiency trade-off limitations of existing structured pruning methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑