基于值感知的Quantization在神经网络训练和推理中的应用
Value-aware Quantization for Training and Inference of Neural Networks
摘要：
我们提出了一种新的值感知量化，它适用于积极降低精度的大部分数据，同时单独处理少量的大数据在高精度，这减少了总的量化误差下非常低的精度。我们提出了新的技术来应用所提出的量化训练和推理。实验表明，我们的3位激活方法（2%的大激活）可以提供与全精度相同的训练精度，同时与最先进的方法相比，ResNet-152和Inception-v3中激活的内存成本显着降低（41.6%和53.7%）。我们的实验还表明，Inception-v3、ResNet-101和DenseNet-121等深度网络可以量化，以进行4位权重和激活（1%的16位数据）的推理，其精度下降幅度在1%的top-1范围内。
英文摘要：
We propose a novel value-aware quantization which applies aggressively reduced precision to the majority of data while separately handling a small amount of large data in high precision, which reduces total quantization errors under very low precision. We present new techniques to apply the proposed quantization to training and inference. The experiments show that our method with 3-bit activations (with 2% of large ones) can give the same training accuracy as full-precision one while offering significant (41.6% and 53.7%) reductions in the memory cost of activations in ResNet-152 and Inception-v3 compared with the state-of-the-art method. Our experiments also show that deep networks such as Inception-v3, ResNet-101 and DenseNet-121 can be quantized for inference with 4-bit weights and activations (with 1% 16-bit data) within 1% top-1 accuracy drop.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
