自动化后端感知后培训Quantization
Automated Backend-Aware Post-Training Quantization
摘要：
量化是降低神经网络部署所需资源、提高性能的关键技术。然而，不同的硬件后端（如x86 CPU、NVIDIA GPU、ARM CPU和加速器）可能需要不同的量化网络实现。这种多样性要求为每个硬件目标构建专门的训练后量化管道，这是一项工程工作，对于开发人员来说往往太大而无法跟上。我们使用一个名为HAGO的自动化训练后量化框架来解决这个问题。HAGO基于用户定义的硬件规范提供了一组通用量化图变换，并实现了一种搜索机制，以找到最佳量化策略，同时满足任何模型的硬件约束。我们观察到HAGO在Intel Xeon Cascade Lake CPU，NVIDIA Tesla T4 GPU，Raspberry Pi4上的ARM Cortex-A CPU上分别实现了2.09倍，1.97倍和2.48倍的加速，同时保持了每种情况下最高的训练后量化精度。
英文摘要：
Quantization is a key technique to reduce the resource requirement and improve the performance of neural network deployment. However, different hardware backends such as x86 CPU, NVIDIA GPU, ARM CPU, and accelerators may demand different implementations for quantized networks. This diversity calls for specialized post-training quantization pipelines to built for each hardware target, an engineering effort that is often too large for developers to keep up with. We tackle this problem with an automated post-training quantization framework called HAGO. HAGO provides a set of general quantization graph transformations based on a user-defined hardware specification and implements a search mechanism to find the optimal quantization strategy while satisfying hardware constraints for any model. We observe that HAGO achieves speedups of 2.09x, 1.97x, and 2.48x on Intel Xeon Cascade Lake CPUs, NVIDIA Tesla T4 GPUs, ARM Cortex-A CPUs on Raspberry Pi4 relative to full precision respectively, while maintaining the highest reported post-training quantization accuracy in each case.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
