arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2205.01508cs.CVcs.AIcs.LG
收藏

通过堆叠设计的基本单元的紧凑神经网络

Compact Neural Networks via Stacking Designed Basic Units

Hong Kong Baptist University(香港浸会大学)

Weichao Lan, Yiu-ming Cheung, Juyong Jiang

摘要:

非结构化剪枝在处理稀疏和不规则的权值时存在局限性。相比之下,结构化修剪可以帮助消除这个缺点,但它需要复杂的标准来确定哪些组件被修剪。为此,本文提出了一种新的构造方法TissueNet,该方法通过独立堆叠设计的基本单元,直接构造具有较少权值参数的紧凑神经网络,不再需要额外的判断标准。给定各种体系结构的基本单元,它们以一定的形式组合和堆叠,以构建紧凑的神经网络。我们在不同的流行骨干中制定TissueNet,以与不同基准数据集上的最先进的修剪方法进行比较。此外,提出了两个新的指标来评估压缩性能。实验结果表明,TissueNet可以达到相当的分类精度,同时节省高达约80%的FLOP和89.7%的参数。也就是说,堆叠基本单元为网络压缩提供了一种新的有前途的方式。

英文摘要:

Unstructured pruning has the limitation of dealing with the sparse and irregular weights. By contrast, structured pruning can help eliminate this drawback but it requires complex criterion to determine which components to be pruned. To this end, this paper presents a new method termed TissueNet, which directly constructs compact neural networks with fewer weight parameters by independently stacking designed basic units, without requiring additional judgement criteria anymore. Given the basic units of various architectures, they are combined and stacked in a certain form to build up compact neural networks. We formulate TissueNet in diverse popular backbones for comparison with the state-of-the-art pruning methods on different benchmark datasets. Moreover, two new metrics are proposed to evaluate compression performance. Experiment results show that TissueNet can achieve comparable classification accuracy while saving up to around 80% FLOPs and 89.7% parameters. That is, stacking basic units provides a new promising way for network compression.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑