arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.07955cs.LG
收藏

重新思考基于补偿的LLM量化中的残差误差

Rethinking Residual Errors in Compensation-based LLM Quantization

Zhejiang University(浙江大学);vivo Mobile Communication Co., Ltd(维沃移动通信有限公司)

Shuaiting Li, Juncan Deng, Kedong Xu, Rongtao Deng, Hong Gu, Minghan Jiang, Haibin Shen, Kejie Huang

AI总结:

本文重新审视残差误差的定义,指出现有方法的校准目标存在次优问题,并引入补偿感知误差以提升LLM量化性能。

摘要:

基于权重补偿的方法通过迭代应用量化和权重补偿来最小化输出误差,最近在量化大型语言模型(LLMs)中表现出色。GPTQ引入了关键技术使此类迭代方法在具有数十亿参数的LLMs中成为现实。GPTAQ通过引入不对称校准过程对齐每个量化层的输出与其全精度对应物,并将残差误差纳入权重补偿框架。本文重新审视残差误差的公式。我们发现现有方法存在次优校准目标:在层内校准过程中,它们将量化输出与补偿权重的输出对齐,而非原始全精度模型的真实输出。因此,我们重新定义目标,使量化模型的输出与全精度模型的原始输出在每一步精确对齐。然后揭示残差误差不仅来自前一层的输出差异,还来自各层内补偿权重与原始权重之间的差异,我们将其命名为'补偿感知误差'。通过继承GPTAQ中的神经分解技术,我们能够高效地将此补偿感知误差纳入权重更新过程中。在各种LLM和量化设置上的广泛实验表明,所提出的改进能够无缝集成到GPTQ和GPTAQ中,显著提升其量化性能。我们的代码在https://github.com/list0830/ResComp上公开可用。

英文摘要:

Methods based on weight compensation, which iteratively apply quantization and weight compensation to minimize the output error, have recently demonstrated remarkable success in quantizing Large Language Models (LLMs). The representative work, GPTQ, introduces several key techniques that make such iterative methods practical for LLMs with billions of parameters. GPTAQ extends this approach by introducing an asymmetric calibration process that aligns the output of each quantized layer with its full-precision counterpart, incorporating a residual error into the weight compensation framework. In this work, we revisit the formulation of the residual error. We identify a sub-optimal calibration objective in existing methods: during the intra-layer calibration process, they align the quantized output with the output from compensated weights, rather than the true output from the original full-precision model. Therefore, we redefine the objective to precisely align the quantized model's output with the original output of the full-precision model at each step. We then reveal that the residual error originates not only from the output difference of the preceding layer but also from the discrepancy between the compensated and original weights within each layer, which we name the 'compensation-aware error'. By inheriting the neuron decomposition technique from GPTAQ, we can efficiently incorporate this compensation-aware error into the weight update process. Extensive experiments on various LLMs and quantization settings demonstrate that our proposed enhancements integrate seamlessly with both GPTQ and GPTAQ, significantly improving their quantization performance. Our code is publicly available at https://github.com/list0830/ResComp.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑