arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2107.06916cs.CVcs.AI
收藏

基于动态编码滤波器融合的图像分类压缩CNN训练

Training Compact CNNs for Image Classification using Dynamic-coded Filter Fusion

Xiamen University(厦门大学);Tencent Youtu Lab(腾讯优图实验室)

Mingbao Lin, Bohong Chen, Fei Chao, Rongrong Ji

摘要:

过滤器修剪的主流方法通常是在计算繁重的预训练模型上强制进行硬编码的重要性估计以选择“重要”过滤器,或者对损失目标施加超参数敏感的稀疏约束以正则化网络训练。在本文中,我们提出了一种新的过滤器修剪方法,称为动态编码过滤器融合(DCFF),以计算经济和无正则化的方式获得紧凑的CNN,用于高效的图像分类。在DCFF中,每个滤波器首先被赋予一个内部相似性分布,并以温度参数作为滤波器代理,在此基础上,提出了一种新的基于Kullback-Leibler散度的动态编码准则来评估滤波器的重要性.与其他方法中简单地保留高分滤波器不同,我们提出了滤波器融合的概念,即,使用指定的代理的加权平均值,作为我们保留的过滤器。当温度参数趋于无穷大时,我们得到了一个单热的相似性分布。因此,每个滤波器的相对重要性可以随着紧凑型CNN的训练而变化,从而导致动态可变的融合滤波器,而不依赖于预训练模型和引入稀疏约束。分类基准上的大量实验表明,我们的DCFF相比同行的优越性。例如,我们的DCFF导出了一个紧凑的VGGNet-16,只有72.77M FLOP和1.06M参数,同时在CIFAR-10上达到了93.47%的top-1精度。在ILSVRC-2012上,获得了具有63.8% FLOP和58.6%参数减少的紧凑ResNet-50,保持了75.60%的top-1精度。我们的代码、更窄的模型和训练日志可以在https://github.com/lmbxmu/DCFF上找到。

英文摘要:

The mainstream approach for filter pruning is usually either to force a hard-coded importance estimation upon a computation-heavy pretrained model to select "important" filters, or to impose a hyperparameter-sensitive sparse constraint on the loss objective to regularize the network training. In this paper, we present a novel filter pruning method, dubbed dynamic-coded filter fusion (DCFF), to derive compact CNNs in a computation-economical and regularization-free manner for efficient image classification. Each filter in our DCFF is firstly given an inter-similarity distribution with a temperature parameter as a filter proxy, on top of which, a fresh Kullback-Leibler divergence based dynamic-coded criterion is proposed to evaluate the filter importance. In contrast to simply keeping high-score filters in other methods, we propose the concept of filter fusion, i.e., the weighted averages using the assigned proxies, as our preserved filters. We obtain a one-hot inter-similarity distribution as the temperature parameter approaches infinity. Thus, the relative importance of each filter can vary along with the training of the compact CNN, leading to dynamically changeable fused filters without both the dependency on the pretrained model and the introduction of sparse constraints. Extensive experiments on classification benchmarks demonstrate the superiority of our DCFF over the compared counterparts. For example, our DCFF derives a compact VGGNet-16 with only 72.77M FLOPs and 1.06M parameters while reaching top-1 accuracy of 93.47% on CIFAR-10. A compact ResNet-50 is obtained with 63.8% FLOPs and 58.6% parameter reductions, retaining 75.60% top-1 accuracy on ILSVRC-2012. Our code, narrower models and training logs are available at https://github.com/lmbxmu/DCFF.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑