arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2010.15979cs.LGcs.ITmath.IT
收藏

量化神经网络的一种贪心算法

A Greedy Algorithm for Quantizing Neural Networks

University of California, San Diego(加利福尼亚大学圣迭戈分校);Halicioglu Data Science Institute(哈利乔卢数据科学学院)

Eric Lybrand, Rayan Saab

摘要:

我们提出了一种新的计算有效的方法来量化预训练的神经网络的权重,该方法足够通用,可以处理多层感知器和卷积神经网络。我们的方法以迭代的方式确定性地量化层,不需要复杂的重新训练。具体来说,我们使用贪婪路径跟踪算法来隐藏每个神经元或隐藏单元。这个简单的算法相当于运行一个动力系统,我们证明当训练数据是高斯的时,该动力系统对于量化单层神经网络(或者,对于量化多层网络的第一层)是稳定的。我们表明,在这些假设下,量化误差随层的宽度衰减,即,过度参数化的程度。我们在多层网络上提供了数值实验,以说明我们的方法在MNIST和CIFAR10数据上的性能,以及使用ImageNet数据量化VGG16网络的性能。

英文摘要:

We propose a new computationally efficient method for quantizing the weights of pre- trained neural networks that is general enough to handle both multi-layer perceptrons and convolutional neural networks. Our method deterministically quantizes layers in an iterative fashion with no complicated re-training required. Specifically, we quantize each neuron, or hidden unit, using a greedy path-following algorithm. This simple algorithm is equivalent to running a dynamical system, which we prove is stable for quantizing a single-layer neural network (or, alternatively, for quantizing the first layer of a multi-layer network) when the training data are Gaussian. We show that under these assumptions, the quantization error decays with the width of the layer, i.e., its level of over-parametrization. We provide numerical experiments, on multi-layer networks, to illustrate the performance of our methods on MNIST and CIFAR10 data, as well as for quantizing the VGG16 network using ImageNet data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑