BaCP：用于在极度稀疏神经网络中保持表示的骨干对比剪枝
BaCP: Backbone Contrastive Pruning for Preserving Representations in Extremely Sparse Neural Networks
- Lahore University of Management Sciences（拉合尔管理科学大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
针对极度稀疏剪枝中的表征崩溃问题，提出BaCP方法，通过对比对齐预训练等模型来正则化嵌入空间，在90种设置下显著提升极度稀疏时的准确率。
AI中文摘要：
在极度稀疏条件下的非结构化剪枝常常遭受表征崩溃，导致准确率急剧下降。为解决这一问题，我们研究了骨干对比剪枝（BaCP），该方法通过将稀疏网络的嵌入空间与预训练、微调及历史快照模型对齐来正则化该空间。基于CAP框架（Xu等人，2022）的对比分解，我们对该方法在多种剪枝标准下提供了严格的匹配预算表征。在90种设置下的评估中，BaCP在标准剪枝失效的极度稀疏场景中显著提升了准确率，而在表征保持完整的场景中与基线接近。
英文摘要：
Unstructured pruning at extreme sparsity often suffers from representational collapse, causing sharp drops in accuracy. To address this, we study Backbone Contrastive Pruning (BaCP), which regularizes the sparse network's embedding space by aligning it with pretrained, fine-tuned, and historical snapshot models. Building on the contrastive decomposition of the CAP framework (Xu et al., 2022), we provide a rigorous matched-budget characterization of this approach across multiple pruning criteria. Evaluated across 90 settings, BaCP improves accuracy substantially in extreme sparsity regimes where standard pruning fails, and is close to baseline where representations remain intact.
