arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1506.02626cs.NEcs.CVcs.LG
收藏

学习有效神经网络的权值和连接

Learning both Weights and Connections for Efficient Neural Networks

Stanford University(斯坦福大学);NVIDIA(英伟达)

Song Han, Jeff Pool, John Tran, William J. Dally

摘要:

神经网络是计算密集型和存储密集型的,使得它们难以部署在嵌入式系统上。此外,传统的网络在训练开始之前就修复了架构;因此,训练无法改善架构。为了解决这些限制,我们描述了一种方法,通过只学习重要的连接,在不影响其准确性的情况下,将神经网络所需的存储和计算减少一个数量级。我们的方法使用三步法修剪冗余连接。首先,我们对网络进行训练,以了解哪些连接是重要的。接下来,我们删除不重要的连接。最后,我们重新训练网络,以微调剩余连接的权重。在ImageNet数据集上,我们的方法将AlexNet的参数数量减少了9倍,从6100万减少到670万,而不会导致准确性损失。使用VGG-16进行的类似实验发现,参数的数量可以减少13倍,从1.38亿减少到1030万,并且仍然没有损失准确性。

英文摘要:

Neural networks are both computationally intensive and memory intensive, making them difficult to deploy on embedded systems. Also, conventional networks fix the architecture before training starts; as a result, training cannot improve the architecture. To address these limitations, we describe a method to reduce the storage and computation required by neural networks by an order of magnitude without affecting their accuracy by learning only the important connections. Our method prunes redundant connections using a three-step method. First, we train the network to learn which connections are important. Next, we prune the unimportant connections. Finally, we retrain the network to fine tune the weights of the remaining connections. On the ImageNet dataset, our method reduced the number of parameters of AlexNet by a factor of 9x, from 61 million to 6.7 million, without incurring accuracy loss. Similar experiments with VGG-16 found that the number of parameters can be reduced by 13x, from 138 million to 10.3 million, again with no loss of accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑