arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.06822cs.AI
收藏

POP:在线结构剪枝实现大基础模型的高效推理

POP: Online Structural Pruning Enables Efficient Inference of Large Foundation Models

Korea Advanced Institute of Science and Technology(韩国科学技术院);University of Tokyo, Tokyo, Japan(东京大学);Tokyo Institute of Technology, Tokyo, Japan(东京技术大学)

Yi Chen, Wonjin Shin, Shuhong Liu, Tho Mai, Jeongmo Lee, Chuanbo Hua, Kun Wang, Jun Liu, Joo-Young Kim

AI总结:

POP通过在线结构剪枝提升大基础模型推理效率,实现动态剪枝与低计算开销。

摘要:

大基础模型(LFMs)通过扩展实现强大性能,但当前的结构剪枝方法在推理过程中得出固定的剪枝决策,忽略了自回归令牌生成中出现的稀疏性模式。在本文中,我们提出了POP(分区引导在线剪枝),一种高效的在线结构剪枝框架,能够实现基于上下文的动态剪枝,且计算开销极低。POP将模型通道划分为保留、候选和剪枝区域,其中预填充定义了粗略的剪枝分区,解码阶段在候选区域生成细粒度的掩码,避免全通道重新评估。粗剪枝分区保留一致重要的权重,而细粒度掩码在解码过程中提供基于上下文的变化。此外,POP是一种轻量级、即插即用的方法,不需要预处理,包括离线校准、重训练或学习预测器。在广泛的大LFMs上进行的评估,包括大型语言模型(LLMs)、专家混合模型(MoEs)和视觉语言模型(VLMs),证明POP在准确率上始终优于现有剪枝方法,同时带来较小的计算开销并最小化推理延迟。

英文摘要:

Large foundation models (LFMs) achieve strong performance through scaling, yet current structural pruning methods derive fixed pruning decisions during inference, overlooking sparsity patterns that emerge in the autoregressive token generation. In this paper, we propose POP (Partition-guided Online Pruning), an efficient online structural pruning framework that enables context-conditioned dynamic pruning with minimal computational overhead. POP partitions model channels into retained, candidate, and pruned regions, where prefilling defines a coarse pruning partition, and the decoding stage generates a fine-grained mask within the candidate region, avoiding full-channel re-evaluation. The coarse pruning partition preserves consistently important weights, while the fine-grained masking provides context-conditioned variation during decoding. Moreover, POP is a lightweight, plug-and-play method that requires no preprocessing, including offline calibration, retraining, or learning predictors. Extensive evaluations across diverse LFMs, including large language models (LLMs), mixture-of-experts models (MoEs), and vision-language models (VLMs), demonstrate that POP consistently delivers higher accuracy than existing pruning approaches while incurring smaller computational overhead and minimizing inference latency.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑