arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1701.08978cs.LGcs.NE
收藏

基于动态不动点的混合低精度深度学习推理

Mixed Low-precision Deep Learning Inference using Dynamic Fixed Point

Intel Labs(英特尔实验室)

Naveen Mellempudi, Abhisek Kundu, Dipankar Das, Dheevatsa Mudigere, Bharat Kaul

摘要:

我们提出了一种基于聚类的量化方法,将预训练的全精度权重转换为三进制权重,对精度的影响最小。此外,我们还将激活限制为8位,从而实现子8位全整数推理流水线。我们的方法使用较小的N个滤波器的集群与一个共同的缩放因子,以最大限度地减少量化损失,同时也最大限度地提高了三进制操作的数量。我们表明,在Resnet-101上,当集群大小为N=4时,可以实现71.8%的TOP-1精度,在最佳全精度结果的6%以内,同时用8位累加替换约85%的乘法。使用相同的方法,4位权重达到76.3%的TOP-1精度,在2%的全精度结果。我们还研究了集群大小对性能和精度的影响,较大的集群大小N=64可以用三元运算代替~98%的乘法,但精度会显著下降,这需要对参数进行微调,并以较低的精度重新训练网络。为了解决这个问题,我们还通过使用全精度权重预初始化网络,训练了具有8位激活和三进制权重的低精度Resnet-50,并在4个额外的epoch内实现了68.9%的TOP-1精度。我们最终的量化模型可以在完整的8位计算管道上运行,与基线全精度模型相比,性能可能提高16倍。

英文摘要:

We propose a cluster-based quantization method to convert pre-trained full precision weights into ternary weights with minimal impact on the accuracy. In addition, we also constrain the activations to 8-bits thus enabling sub 8-bit full integer inference pipeline. Our method uses smaller clusters of N filters with a common scaling factor to minimize the quantization loss, while also maximizing the number of ternary operations. We show that with a cluster size of N=4 on Resnet-101, can achieve 71.8% TOP-1 accuracy, within 6% of the best full precision results while replacing ~85% of all multiplications with 8-bit accumulations. Using the same method with 4-bit weights achieves 76.3% TOP-1 accuracy which within 2% of the full precision result. We also study the impact of the size of the cluster on both performance and accuracy, larger cluster sizes N=64 can replace ~98% of the multiplications with ternary operations but introduces significant drop in accuracy which necessitates fine tuning the parameters with retraining the network at lower precision. To address this we have also trained low-precision Resnet-50 with 8-bit activations and ternary weights by pre-initializing the network with full precision weights and achieve 68.9% TOP-1 accuracy within 4 additional epochs. Our final quantized model can run on a full 8-bit compute pipeline, with a potential 16x improvement in performance compared to baseline full-precision models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑