arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2207.03088cs.LGcs.AI
收藏

Quantization后训练的注意轮

Attention Round for Post-Training Quantization

Institute of Microelectronics, Chinese Academy of Sciences(中国科学院微电子研究所);University of Chinese Academy of Sciences(中国科学院大学)

Huabin Diao, Gongyan Li, Shaoyun Xu, Yuexing Hao

摘要:

目前,神经网络模型的量化方法主要分为后训练量化(PTQ)和量化感知训练(QAT)。后训练量化只需要一小部分数据就可以完成量化过程,但其量化模型的性能不如量化感知训练。本文提出了一种新的量化方法称为注意轮。该方法使参数w有机会被映射到所有可能的量化值,而不仅仅是量化过程中w附近的两个量化值。被映射到不同量化值的概率与量化值与w之间的距离负相关,并以高斯函数衰减。此外,本文以有损编码长度为度量,为模型的不同层分配位宽,解决了混合精度量化问题,有效避免了求解组合优化问题。本文还对不同的模型进行了定量实验,实验结果证实了该方法的有效性。对于ResNet 18和MobileNetV 2,本文提出的后训练量化方法只需要1,024个训练数据和10分钟就可以完成量化过程,可以达到与量化感知训练相当的量化性能。

英文摘要:

At present, the quantification methods of neural network models are mainly divided into post-training quantization (PTQ) and quantization aware training (QAT). Post-training quantization only need a small part of the data to complete the quantification process, but the performance of its quantitative model is not as good as the quantization aware training. This paper presents a novel quantification method called Attention Round. This method gives parameters w the opportunity to be mapped to all possible quantized values, rather than just the two quantized values nearby w in the process of quantization. The probability of being mapped to different quantified values is negatively correlated with the distance between the quantified values and w, and decay with a Gaussian function. In addition, this paper uses the lossy coding length as a measure to assign bit widths to the different layers of the model to solve the problem of mixed precision quantization, which effectively avoids to solve combinatorial optimization problem. This paper also performs quantitative experiments on different models, the results confirm the effectiveness of the proposed method. For ResNet18 and MobileNetV2, the post-training quantization proposed in this paper only require 1,024 training data and 10 minutes to complete the quantization process, which can achieve quantization performance on par with quantization aware training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑