Feather：一种优雅的高效DNN稀疏化解决方案
Feather: An Elegant Solution to Effective DNN Sparsification
AI总结：
提出Feather稀疏训练模块，利用直通估计器、新阈值算子和梯度缩放技术，在标准训练中实现高效稳健的DNN稀疏化，在ImageNet上以ResNet-50达到最先进的Top-1准确率。
摘要：
神经网络剪枝是一种日益流行的生成紧凑高效模型的方法，适用于资源受限的环境，同时保持高性能。虽然剪枝可以通过多周期训练和微调过程来执行，但近期的趋势是在标准训练过程中包含稀疏化过程。为此，我们引入了Feather，一种高效的稀疏训练模块，其核心利用强大的直通估计器，并配以新的阈值算子和梯度缩放技术，从而实现稳健、开箱即用的稀疏化性能。Feather的有效性和适应性通过在CIFAR数据集上的多种架构得到展示，而在ImageNet上，使用ResNet-50架构，它达到了最先进的Top-1验证准确率，显著超越了现有方法，包括更复杂和计算量更大的方法。代码公开于https://github.com/athglentis/feather。
英文摘要：
Neural Network pruning is an increasingly popular way for producing compact and efficient models, suitable for resource-limited environments, while preserving high performance. While the pruning can be performed using a multi-cycle training and fine-tuning process, the recent trend is to encompass the sparsification process during the standard course of training. To this end, we introduce Feather, an efficient sparse training module utilizing the powerful Straight-Through Estimator as its core, coupled with a new thresholding operator and a gradient scaling technique, enabling robust, out-of-the-box sparsification performance. Feather's effectiveness and adaptability is demonstrated using various architectures on the CIFAR dataset, while on ImageNet it achieves state-of-the-art Top-1 validation accuracy using the ResNet-50 architecture, surpassing existing methods, including more complex and computationally heavy ones, by a considerable margin. Code is publicly available at https://github.com/athglentis/feather .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
