Quantune：使用极端梯度提升快速部署的卷积神经网络后训练Quantization
Quantune: Post-training Quantization of Convolutional Neural Networks using Extreme Gradient Boosting for Fast Deployment
摘要：
为了对一系列资源受限的目标采用卷积神经网络（CNN），有必要通过执行量化来压缩CNN模型，由此将精度表示转换为较低比特表示。为了克服诸如训练数据集的敏感性、高计算要求和大时间消耗之类的问题，已经提出了不需要再训练的后训练量化方法。此外，为了在不进行再训练的情况下补偿精度下降，以前关于后训练量化的研究提出了几种补充方法：校准、方案、裁剪、粒度和混合精度。为了生成具有最小误差的量化模型，有必要研究这些方法的所有可能组合，因为它们中的每一个都是互补的，并且CNN模型具有不同的特性。然而，穷举或启发式搜索要么太耗时，要么是次优的。为了克服这一挑战，我们提出了一种称为Quantune的自动调谐器，它构建了一个梯度树提升模型，以加速搜索量化配置并减少量化误差。我们评估和比较Quantune与随机，网格，和遗传算法。实验结果表明，Quantune将量化的搜索时间减少了约36.5倍，在六个CNN模型中的准确性损失为0.07~0.65%，包括脆弱的模型（MobileNet，SqueezeNet和ShuffleNet）。为了支持多个目标并采用不断发展的量化工作，Quantune作为一个开源项目在一个成熟的深度学习编译器上实现。
英文摘要：
To adopt convolutional neural networks (CNN) for a range of resource-constrained targets, it is necessary to compress the CNN models by performing quantization, whereby precision representation is converted to a lower bit representation. To overcome problems such as sensitivity of the training dataset, high computational requirements, and large time consumption, post-training quantization methods that do not require retraining have been proposed. In addition, to compensate for the accuracy drop without retraining, previous studies on post-training quantization have proposed several complementary methods: calibration, schemes, clipping, granularity, and mixed-precision. To generate a quantized model with minimal error, it is necessary to study all possible combinations of the methods because each of them is complementary and the CNN models have different characteristics. However, an exhaustive or a heuristic search is either too time-consuming or suboptimal. To overcome this challenge, we propose an auto-tuner known as Quantune, which builds a gradient tree boosting model to accelerate the search for the configurations of quantization and reduce the quantization error. We evaluate and compare Quantune with the random, grid, and genetic algorithms. The experimental results show that Quantune reduces the search time for quantization by approximately 36.5x with an accuracy loss of 0.07 ~ 0.65% across six CNN models, including the fragile ones (MobileNet, SqueezeNet, and ShuffleNet). To support multiple targets and adopt continuously evolving quantization works, Quantune is implemented on a full-fledged compiler for deep learning as an open-sourced project.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
