arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2102.13188cs.NE
收藏

一种使用基于能量的模型修剪深度神经网络的框架

A Framework For Pruning Deep Neural Networks Using Energy-Based Models

Hojjat Salehinejad, Shahrokh Valaee

摘要:

典型的深度神经网络(DNN)具有大量的可训练参数。选择具有适当容量的网络是具有挑战性的,并且通常训练具有过量容量的较大网络。修剪是减少DNN中参数数量的一种既定方法。在本文中,我们提出了一个基于种群的全局优化方法修剪DNN的框架。该框架可以使用任何修剪目标函数。作为一个案例研究,我们提出了一个简单而有效的目标函数的概念,基于能量的模型。我们在ResNets、AlexNet和SqueezeNet上针对CIFAR-10和CIFAR-100数据集进行的实验显示,可训练参数的修剪率超过$50\%$,分类准确度分别下降约$<5\%$和$<1\%$。

英文摘要:

A typical deep neural network (DNN) has a large number of trainable parameters. Choosing a network with proper capacity is challenging and generally a larger network with excessive capacity is trained. Pruning is an established approach to reducing the number of parameters in a DNN. In this paper, we propose a framework for pruning DNNs based on a population-based global optimization method. This framework can use any pruning objective function. As a case study, we propose a simple but efficient objective function based on the concept of energy-based models. Our experiments on ResNets, AlexNet, and SqueezeNet for the CIFAR-10 and CIFAR-100 datasets show a pruning rate of more than $50\%$ of the trainable parameters with approximately $<5\%$ and $<1\%$ drop of Top-1 and Top-5 classification accuracy, respectively.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑