arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.15960cs.SDeess.AS
收藏

探索通道稀疏性对声学场景分类神经网络剪枝的影响

Exploring the Effects of Channel Sparsity on Neural Network Pruning for Acoustic Scene Classification

Xi'an Jiaotong-Liverpool University(西交利物浦大学)

Yiqiang Cai, Shengchen Li

AI总结:

该研究针对声学场景分类任务,探索通道稀疏性对神经网络剪枝的影响,提出权重偏度指标量化通道稀疏性及平衡准确率与复杂度的剪枝性能对比方法,验证了高通道稀疏性可兼顾压缩率与准确率,且MobileNets从中获益更显著。

摘要:

声学场景分类(Acoustic Scene Classification, ASC)算法通常需部署在资源受限的系统中。现有研究通过剪枝部分组件(如神经网络中的通道)来降低ASC算法的复杂度。实践中,神经网络常通过稀疏化训练,以找出不重要的通道并进一步剪枝。然而,目前针对通道稀疏性对神经网络剪枝的影响的探索仍较为有限。为充分发挥剪枝在ASC中的优势,并确保模型性能稳定,我们需要更深入地理解通道稀疏化及其效应。本文研究了将被剪枝的卷积神经网络所习得的内部权重,探讨了如何利用这些权重构建一种新型量化通道稀疏性的指标——权重偏度(Weight Skewness, WS)。我们还提出了一种对比不同剪枝方法性能的新方法,该方法可平衡准确率与复杂度之间的权衡。实验结果表明:1)对模型施加更高的通道稀疏性,可在维持可接受准确率水平的同时实现更高的压缩率;2)剪枝方法的选择对上述结论1的影响极小;3)与VGGNets和ResNets相比,MobileNets从通道稀疏化中获得的增益更为显著。

英文摘要:

Acoustic Scene Classification (ASC) algorithms are usually expected to be deployed in resource-constrained systems. Existing works reduce the complexity of ASC algorithms by pruning some components, e.g. pruning channels in neural network. In practice, neural networks are often trained with sparsification such that unimportant channels can be found and further pruned. However, little efforts have been made to explore the the impact of channel sparsity on neural network pruning. To fully utilize the benefits of pruning for ASC, and to make sure the model performs consistently, we need a more profound comprehension of channel sparsification and its effects. This paper examines the internal weights acquired by convolutional neural networks that will undergone pruning. The study discusses how these weights can be utilized to create a novel metric, Weight Skewness (WS), for quantifying the sparsity of channels. We also provide a new approach to compare the performance of different pruning methods, which balances the trade-off between accuracy and complexity. The experiment results demonstrate that 1) applying higher channel sparsity to models can achieve greater compression rates while maintaining acceptable levels of accuracy; 2) the selection of pruning method has little influence on result 1); 3) MobileNets exhibit more significant benefits from channel sparsification than VGGNets and ResNets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑