arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.16121cs.CVcs.AI
收藏

面向重参数化模型的精确训练后量化

Towards Accurate Post-training Quantization for Reparameterized Models

Zhejiang University(浙江大学);Shanghai Jiao Tong University(上海交通大学)

Luoming Zhang, Yefei He, Wen Fei, Zhenyu Lou, Weijia Wu, YangWei Ying, Hong Zhou

AI总结:

本文提出 RepAPQ 框架,通过 MAE 度量、量化保护重参数化和跨块校准缓解离群值影响,在 8 位和 6 位 PTQ 上分别提升约 1% 和 2% 的精度。

摘要:

模型重参数化是一种被广泛接受的技术,可在不牺牲性能的情况下提高推理速度。然而,当前的训练后量化(Post-training Quantization, PTQ)方法在应用于重参数化模型时,往往会导致显著的精度下降。这主要是由通道特定和样本特定的离群值引起的,这些离群值仅出现在特定样本和特定通道上,并影响量化参数的选择。为了解决这一问题,我们提出了 RepAPQ,一个能够保持量化重参数化模型精度的新型框架。与以往使用均方误差(Mean Squared Error, MSE)作为度量标准的框架不同,我们利用平均绝对误差(Mean Absolute Error, MAE)来减轻离群值对量化参数的影响。我们的框架包含两个主要组件:量化保护重参数化(Quantization Protecting Reparameterization)和跨块校准(Across-block Calibration)。为了实现有效校准,量化保护重参数化将多个分支合并为一个带有仿射层的单一卷积。在训练过程中,仿射层加速收敛并放大卷积的输出,以更好地适应带有离群值的样本。此外,跨块校准利用阶段输出的度量作为监督,以解决 MAE 引入的梯度问题,并增强与量化参数相关的层间相关性。综合实验证明了 RepAPQ 在各种模型和任务上的有效性。我们的框架在 8 位 PTQ 上比以往方法高出约 1%,在 6 位 PTQ 上高出约 2%,展示了其优越的性能。代码可在 \url{https://github.com/ilur98/DLMC-QUANT} 获取。

英文摘要:

Model reparameterization is a widely accepted technique for improving inference speed without compromising performance. However, current Post-training Quantization (PTQ) methods often lead to significant accuracy degradation when applied to reparameterized models. This is primarily caused by channel-specific and sample-specific outliers, which appear only at specific samples and channels and impact on the selection of quantization parameters. To address this issue, we propose RepAPQ, a novel framework that preserves the accuracy of quantized reparameterization models. Different from previous frameworks using Mean Squared Error (MSE) as a measurement, we utilize Mean Absolute Error (MAE) to mitigate the influence of outliers on quantization parameters. Our framework comprises two main components: Quantization Protecting Reparameterization and Across-block Calibration. For effective calibration, Quantization Protecting Reparameterization combines multiple branches into a single convolution with an affine layer. During training, the affine layer accelerates convergence and amplifies the output of the convolution to better accommodate samples with outliers. Additionally, Across-block Calibration leverages the measurement of stage output as supervision to address the gradient problem introduced by MAE and enhance the interlayer correlation with quantization parameters. Comprehensive experiments demonstrate the effectiveness of RepAPQ across various models and tasks. Our framework outperforms previous methods by approximately 1\% for 8-bit PTQ and 2\% for 6-bit PTQ, showcasing its superior performance. The code is available at \url{https://github.com/ilur98/DLMC-QUANT}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑