批量自适应剪枝：面向语言推理模型的周期性神经元激活感知权重剪枝
Batch-wise Adaptive Pruning: Periodic Neuron Activation-Aware Weight Pruning for Language Reasoning Model
浏览论文内容
中文总结 AI 辅助
本研究针对语言推理模型批量推理下自适应剪枝性能下降的问题，提出带周期性top-k选择和激活记忆的无训练剪枝方法，在DeepSeek-R1-Distill-Qwen-7B上实现39.7个百分点准确率提升与1.40倍推理加速。
中文摘要 AI 辅助
大型推理模型（Large Reasoning Models, LRMs）通过扩展思维链生成在复杂任务上取得了优异性能，但推理阶段会产生巨大的计算成本。在生产环境中，批量推理对于高吞吐量至关重要，但我们评估的现有无训练自适应剪枝方法在该场景下性能严重下降。由于批量需共享单一剪枝掩码，这些方法会聚合不同样本的激活值，再应用基于阈值的选择；而在未聚合激活值上离线校准的阈值不再匹配聚合后的分布，导致实际稀疏率漂移，推理任务的准确性在批量推理下大幅下降。本研究提出一种专门针对LRMs批量推理设计的无训练自适应剪枝方法，包含两个核心组件：其一，用聚合重要性分数上的周期性top-k选择替代基于阈值的选择，该方法不受聚合引发的激活分布偏移影响，且每更新周期仅执行一次选择，而非每个 token 都执行，从而保留加速效果；其二，基于重要神经元在长推理生成过程中会周期性重新激活的观察，引入激活记忆机制，在各更新阶段累积重要性，以保留重复出现的神经元。在多种推理基准上的实验表明，我们的方法在批量大小为4、目标稀疏率50%的设置下，于DeepSeek-R1-Distill-Qwen-7B模型上的平均准确率较之前最优的自适应剪枝方法提升了39.7个百分点，且在实际稀疏率50%时实现了比密集推理快1.40倍的加速效果。
英文摘要
Large Reasoning Models (LRMs) achieve strong performance on complex tasks through extended chain-of-thought generation, but incur substantial computational costs during inference. In production settings, batched inference is essential for high throughput, yet the existing training-free adaptive pruning methods we evaluate severely degrade in this regime. Because a batch must share a single pruning mask, these methods aggregate activations across samples and then apply threshold-based selection; the threshold, calibrated offline on unaggregated activations, no longer matches the aggregated distribution, so the realized sparsity ratio drifts and accuracy on reasoning tasks collapses under batched inference. In this work, we propose a training-free adaptive pruning method designed specifically for batched inference in LRMs, built on two components. First, we replace threshold-based selection with periodic top-k selection over the aggregated importance scores, which is unaffected by the shift that aggregation induces in the activation distribution, and which runs selection once per update period rather than at every token, preserving the speedup. Second, based on the observation that important neurons re-fire periodically during long reasoning generation, we introduce an activation memory that accumulates importance across update phases so that recurring neurons are retained. Experiments on diverse reasoning benchmarks demonstrate that our method outperforms the previous state-of-the-art adaptive pruning method by 39.7 percentage points in average accuracy at batch size 4 with 50% target sparsity on DeepSeek-R1-Distill-Qwen-7B, and reaches 1.40x speedup over dense inference at 50% actual sparsity.
发表机构
- The University of Tokyo（东京大学）
机构由 AI 辅助整理，请以论文原文为准。
