arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1602.02505cs.LGcs.NE
收藏

二值化神经网络

Binarized Neural Networks

Itay Hubara, Daniel Soudry, Ran El Yaniv

摘要:

介绍了一种训练二值化神经网络(BNN)的方法--在运行时和在训练时计算参数梯度时使用二值权值和激活值的神经网络。我们进行了两组实验,每组基于不同的框架,即Torch 7和Theano,我们在MNIST,CIFAR-10和SVHN上训练BNN,并取得了接近最先进的结果。在前向过程中,BNN大大减少了内存大小和访问,并将大多数算术运算替换为逐位运算,这可能会大大提高功耗效率。最后但并非最不重要的是,我们编写了一个二进制矩阵乘法GPU内核,使用它可以运行我们的MNIST BNN比使用未优化的GPU内核快7倍,而不会损失任何分类精度。用于训练和运行我们的BNN的代码是可用的。

英文摘要:

We introduce a method to train Binarized Neural Networks (BNNs) - neural networks with binary weights and activations at run-time and when computing the parameters' gradient at train-time. We conduct two sets of experiments, each based on a different framework, namely Torch7 and Theano, where we train BNNs on MNIST, CIFAR-10 and SVHN, and achieve nearly state-of-the-art results. During the forward pass, BNNs drastically reduce memory size and accesses, and replace most arithmetic operations with bit-wise operations, which might lead to a great increase in power-efficiency. Last but not least, we wrote a binary matrix multiplication GPU kernel with which it is possible to run our MNIST BNN 7 times faster than with an unoptimized GPU kernel, without suffering any loss in classification accuracy. The code for training and running our BNNs is available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑