arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2002.00623cs.NEcs.CV
收藏

预训练神经网络中神经网络连接权值的指数离散化

Exponential discretization of weights of neural network connections in pre-trained neural networks

Center of Optical Neural Technologies, Scientific Research Institute for System Analysis RAS(俄罗斯科学院系统分析研究所光学神经技术中心);Peoples Friendship University of Russia (RUDN University)(俄罗斯人民友谊大学(RUDN大学))

Magomed Yu. Malsagov, Emil M. Khayrov, Maria M. Pushkareva, Iakov M. Karandashev

摘要:

为了减少随机存取存储器(RAM)的要求,并提高识别算法的速度,我们考虑训练神经网络的权重离散化问题。我们表明,指数离散化是优选的线性离散化,因为它允许一个达到相同的精度时,位数为1或2 less. The质量的神经网络VGG-16已经是令人满意的(top5精度69%)的情况下,3位指数离散化。ResNet 50神经网络在4位时显示出84%的前5名准确率。其他神经网络在5位上表现相当好(Xception,Inception-v3和MobileNet-v2 top5的准确率分别为87%,90%和77%)。当比特数较少时,精度迅速下降。

英文摘要:

To reduce random access memory (RAM) requirements and to increase speed of recognition algorithms we consider a weight discretization problem for trained neural networks. We show that an exponential discretization is preferable to a linear discretization since it allows one to achieve the same accuracy when the number of bits is 1 or 2 less. The quality of the neural network VGG-16 is already satisfactory (top5 accuracy 69%) in the case of 3 bit exponential discretization. The ResNet50 neural network shows top5 accuracy 84% at 4 bits. Other neural networks perform fairly well at 5 bits (top5 accuracies of Xception, Inception-v3, and MobileNet-v2 top5 were 87%, 90%, and 77%, respectively). At less number of bits, the accuracy decreases rapidly.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑