arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.04013cs.CL
收藏

RUQuant: 向大语言模型的均匀量化精修迈进

RUQuant: Towards Refining Uniform Quantization for Large Language Models

Dalian University of Technology(大连理工大学);Peking University(北京大学);Macao Polytechnic University(澳门理工大学)

Han Liu, Haotian Gao, Changya Li, Feng Zhang, Xiaotong Zhang, Wei Wang, Hong Yu

AI总结:

本文提出RUQuant方法,通过理论分析和两阶段正交变换,解决激活分布非均匀导致的量化精度下降问题,在不需微调的情况下实现高精度量化。

摘要:

随着大语言模型(LLMs)规模和复杂度的增加,部署效率面临显著挑战,特别是在资源受限条件下。后训练量化(PTQ)作为一种实用解决方案,通过压缩模型而不需重新训练。尽管现有方法专注于权重和激活的均匀量化方案,但因激活分布的非均匀性导致精度显著下降。本文从理论角度出发,基于Lloyd-Max最优性条件重新审视激活量化问题。我们发现核心问题在于量化区间内激活分布的非均匀性,导致Lloyd-Max准则下的最优量化点偏离区间中点。为解决此问题,我们提出两阶段正交变换方法RUQuant。第一阶段将激活分为块,每个块通过复合正交矩阵映射到均匀采样的目标向量,该矩阵由Householder反射和Givens旋转构建。第二阶段通过Transformer输出差异进一步微调全局Householder反射以最小化量化误差。实验结果表明,我们的方法在不需模型微调的情况下实现了接近全精度的量化性能:RUQuant在W6A6和W4A4量化下,对于13B LLM实现99.8%和97%的全精度准确率,耗时约一分钟。微调变体进一步提高了精度,展示了方法的有效性和可扩展性。

英文摘要:

The increasing size and complexity of large language models (LLMs) have raised significant challenges in deployment efficiency, particularly under resource constraints. Post-training quantization (PTQ) has emerged as a practical solution by compressing models without requiring retraining. While existing methods focus on uniform quantization schemes for both weights and activations, they often suffer from substantial accuracy degradation due to the non-uniform nature of activation distributions. In this work, we revisit the activation quantization problem from a theoretical perspective grounded in the Lloyd-Max optimality conditions. We identify the core issue as the non-uniform distribution of activations within the quantization interval, which causes the optimal quantization point under the Lloyd-Max criterion to shift away from the midpoint of the interval. To address this issue, we propose a two-stage orthogonal transformation method, RUQuant. In the first stage, activations are divided into blocks. Each block is mapped to uniformly sampled target vectors using composite orthogonal matrices, which are constructed from Householder reflections and Givens rotations. In the second stage, a global Householder reflection is fine-tuned to further minimize quantization error using Transformer output discrepancies. Empirical results show that our method achieves near-optimal quantization performance without requiring model fine-tuning: RUQuant achieves 99.8% of full-precision accuracy with W6A6 and 97% with W4A4 quantization for a 13B LLM, within approximately one minute. A fine-tuned variant yields even higher accuracy, demonstrating the effectiveness and scalability of our approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑