量化深度卷积网络以实现高效推理：白皮书
Quantizing deep convolutional networks for efficient inference: A whitepaper
摘要：
我们提出了一个概述的技术量化卷积神经网络的推理与整数权重和激活。对于各种各样的CNN架构，权重的每通道量化和激活的每层量化到8位精度后训练产生在浮点网络的2%内的分类准确度。通过将权重量化为8位，即使不支持8位算法，也可以将模型大小减少4倍。这可以通过简单的后训练量化权重来实现。我们在CPU和DSP上对量化网络的浮点数进行了基准测试，并观察到量化实现与CPU上的浮点数相比的2倍-3倍的加速比。在具有固定点SIMD功能的专用处理器上，如采用HVX的Qualcomm QDSP，可以观察到高达10倍的速度提升。量化感知训练可以提供进一步的改进，在8位精度下将浮点差距缩小到1%。量化感知训练还允许将权重的精度降低到4位，精度损失范围从2%到10%，对于较小的网络，精度损失更高。我们介绍了TensorFlow和TensorFlowLite中用于量化卷积网络的工具，并回顾了量化感知训练的最佳实践，以获得量化权重和激活的高精度。我们建议将权重的每通道量化和激活的每层量化作为硬件加速和内核优化的首选量化方案。我们还建议，未来的处理器和硬件加速器优化推理支持精度为4，8和16位。
英文摘要：
We present an overview of techniques for quantizing convolutional neural networks for inference with integer weights and activations. Per-channel quantization of weights and per-layer quantization of activations to 8-bits of precision post-training produces classification accuracies within 2% of floating point networks for a wide variety of CNN architectures. Model sizes can be reduced by a factor of 4 by quantizing weights to 8-bits, even when 8-bit arithmetic is not supported. This can be achieved with simple, post training quantization of weights.We benchmark latencies of quantized networks on CPUs and DSPs and observe a speedup of 2x-3x for quantized implementations compared to floating point on CPUs. Speedups of up to 10x are observed on specialized processors with fixed point SIMD capabilities, like the Qualcomm QDSPs with HVX. Quantization-aware training can provide further improvements, reducing the gap to floating point to 1% at 8-bit precision. Quantization-aware training also allows for reducing the precision of weights to four bits with accuracy losses ranging from 2% to 10%, with higher accuracy drop for smaller networks.We introduce tools in TensorFlow and TensorFlowLite for quantizing convolutional networks and review best practices for quantization-aware training to obtain high accuracy with quantized weights and activations. We recommend that per-channel quantization of weights and per-layer quantization of activations be the preferred quantization scheme for hardware acceleration and kernel optimization. We also propose that future processors and hardware accelerators for optimized inference support precisions of 4, 8 and 16 bits.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
