发表机构
KAIST AI; New York University Abu Dhabi（韩国科学技术院人工智能学院; 纽约大学阿布扎比分校）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对ViTs剪枝的精度下降或需再训练的问题，提出DVBP + OB²C方法，结合随机矩阵理论去噪与最优大脑偏差补偿，在50% MLP剪枝时保留超90%原Top-1精度，优于VBP。
AI 中文摘要
视觉Transformer（ViTs）达到了当前最优性能，但计算开销巨大，限制了其在边缘设备上的部署。结构化剪枝是降低该开销的关键策略，但现有方法常出现严重的精度下降，或需要昂贵的再训练。最近，方差剪枝（VBP）通过基于激活方差选择神经元提供了有前景的范式，但它受限于有限样本激活协方差中的统计噪声，且依赖仅偏差更新，无法完全解决结构重建误差。为解决这些局限，我们提出结合最优大脑偏差补偿的去噪方差剪枝（DVBP + OB²C）。我们利用随机矩阵理论从激活协方差谱中过滤噪声以实现鲁棒的神经元选择，并从数学上证明，将均值漂移补偿集成到最优大脑压缩目标中，可将层Hessian精确简化为激活协方差矩阵，这使得能使用选择时收集的相同统计量对剩余权重进行最优闭式更新。在DeiT、Swin和ConvNeXt架构上的大量实验表明，DVBP + OB²C实现了当前最优的无训练性能；在50%的MLP剪枝率下，它在Small和Base变体上保留了超过90%的原始Top-1精度，在ConvNeXt-T上比VBP高出29.46%，在Swin-S上高出7.33%。代码可在该网址获取。
英文摘要
Vision Transformers (ViTs) achieve state-of-the-art performance but carry massive computational overhead that restricts edge deployment. Although structural pruning has emerged as a key strategy to reduce these costs, existing methods often suffer from severe accuracy degradation or require expensive retraining. Recently, Variance-Based Pruning (VBP) introduced a promising paradigm by selecting neurons based on activation variance; however, it remains limited by statistical noise in finite-sample activation covariance and reliance on bias-only updates that cannot fully account for structural reconstruction error. To address these limitations, we introduce Denoised Variance-Based Pruning with Optimal Brain Bias Compensation (DVBP + OB$^2$C). We leverage random matrix theory to filter noise from the activation covariance spectrum for robust neuron selection and mathematically prove that integrating mean-shift compensation into the Optimal Brain Compression objective reduces the layer-wise Hessian exactly to the activation covariance matrix. This enables an optimal, closed-form update of the remaining weights using the same statistics gathered for selection. Extensive experiments on DeiT, Swin, and ConvNeXt architectures demonstrate that DVBP + OB$^2$C achieves state-of-the-art training-free performance; at 50% MLP pruning, it retains over 90% of the original Top-1 accuracy on Small and Base variants, outperforming VBP by up to 29.46% (ConvNeXt-T) and 7.33% (Swin-S). The code is available at: https://github.com/geontackee/DVBP_OB2C.
CommentsAccepted to ECCV 2026
