稀疏自适应锐度感知最小化
Sparsity-Adaptive Sharpness-Aware Minimization
浏览论文内容
中文总结 AI 辅助
提出稀疏自适应锐度感知最小化（SA-SAM），通过保持平均绝对扰动不变推导稀疏相关扰动半径，在CIFAR和ImageNet损坏基准上于80-90%稀疏度下提升鲁棒性并保持准确率。
中文摘要 AI 辅助
在真实世界环境中部署深度神经网络，需要模型既紧凑又对常见损坏具有鲁棒性。然而，在部署相关的高稀疏度下，标准的剪枝流程往往会降低损坏鲁棒性，而现有的锐度感知训练/剪枝方法提供的鲁棒性提升有限。我们通过引入稀疏自适应锐度感知最小化（SA-SAM）来解决这个问题，该方法通过保持平均绝对扰动（一种基于ℓ1的代理）在稀疏度增加时近似不变，来推导出依赖于稀疏度的SAM/ASAM扰动半径。作为一个简单的补充选项，我们评估了幅度加权海森矩阵（MWH），它源自二阶移除路径分析，产生的重要性与Diag(F)_i |w_i|成正比，其中Diag(F)是我们实现中用作曲率代理的对角经验Fisher。在CIFAR-10-C、CIFAR-100-C和ImageNet-100-C上，我们的方法在80%至90%的稀疏度下，相比所考虑的剪枝基线，实现了更强的损坏鲁棒性，同时保持了干净准确性。我们还通过报告在部署相关稀疏度下稀疏执行时的实测推理吞吐量，来量化鲁棒性与吞吐量之间的权衡。
英文摘要
Deploying deep neural networks in real-world settings requires models that are both compact and robust to common corruptions. However, at deployment-relevant high sparsity, standard pruning pipelines often degrade corruption robustness, and existing sharpness-aware training/pruning approaches provide limited robustness gains. We address this issue by introducing Sparsity-Adaptive Sharpness-Aware Minimization (SA-SAM), which derives a sparsity-dependent SAM/ASAM perturbation radius by keeping the mean absolute perturbation (an $\ell_1$-based proxy) approximately invariant as sparsity increases. As a simple complementary option, we evaluate Magnitude-Weighted Hessian (MWH), derived from a second-order removal-path analysis, yielding an importance proportional to $\mathrm{Diag}(F)_i\,|w_i|$, where $\mathrm{Diag}(F)$ is the diagonal empirical Fisher used as a curvature proxy in our implementation. Across CIFAR-10-C, CIFAR-100-C, and ImageNet-100-C, our approach achieved stronger corruption robustness than the considered pruning baselines at 80--90\% sparsity, while preserving clean accuracy. We additionally quantify the robustness--throughput trade-off by reporting measured inference throughput under sparse execution at deployment-relevant sparsity levels.
发表机构
- Gifu University（岐阜大学）
机构由 AI 辅助整理，请以论文原文为准。
