arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新，休息一下吧，周末愉快~~
arXiv 2605.05693cs.AIcs.LG
收藏

具有显著性感知的正则化量化校准用于大语言模型

Saliency-Aware Regularized Quantization Calibration for Large Language Models

University of Science and Technology of China（中国科学技术大学）；University College London（伦敦大学学院）；Shanghai University of Finance and Economics（上海财经大学）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；University of Copenhagen（哥本哈根大学）；Imperial College London（伦敦帝国学院）；Technical University of Denmark（丹麦技术大学）；Peking University（北京大学）

Yanlong Zhao, Xiaoyuan Cheng, Huihang Liu, Baihua He, Xinyu Zhang, Harrison Bo Hua Zhu, Wenlong Chen, Li Zeng, Zhuo Sun

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出SARQC，通过引入显著性感知正则化改进量化校准，提升大语言模型的泛化能力，实验表明在密集和专家混合模型中提升了困惑度和零样本准确率。

摘要：

后训练量化（PTQ）是一种在内存和延迟约束下部署大语言模型（LLMs）的有效方法。现有PTQ方法通过最小化预定义校准数据集上的层间重建误差确定量化参数，通常通过缩放搜索或Gram基方法优化。然而，从泛化风险的角度来看，基于有限或不具代表性的校准数据的重建误差可能使量化权重偏离原始浮点权重，从而降低下游性能。为了解决这个问题，我们提出了正则化量化校准（RQC），这是一个统一的框架，通过添加显式控制权重偏离原始权重的正则化项来增强标准PTQ目标。我们进一步将此框架推广以纳入显著性感知正则化，得到显著性感知正则化量化校准（SARQC）。所提出的正则化鼓励量化权重在校准过程中保持接近原始权重，从而在推理时提高泛化能力。SARQC无缝集成到现有的PTQ流程中，并在统一的公式下增强缩放搜索和Gram基方法。在密集和专家混合LLM上的广泛实验显示，SARQC在困惑度和零样本准确率上表现出一致的改进，而无需引入额外的推理开销。

英文摘要：

Post-training quantization (PTQ) is an effective approach for deploying large language models (LLMs) under memory and latency constraints. Most existing PTQ methods determine quantization parameters by minimizing a layer-wise reconstruction error on a predetermined calibration dataset, typically optimized via either scale search or Gram-based methods. However, from the perspective of generalization risk, existing PTQ calibration objectives based solely on empirical reconstruction error over limited or unrepresentative calibration data may move the quantized weights away from the original floating-point weights, potentially degrading downstream performance. To address this issue, we propose \emph{Regularized Quantization Calibration} (RQC), a unified framework that augments standard PTQ objectives with a regularizer that explicitly controls weight deviation from the original weights. We further generalize this framework to incorporate a saliency-aware regularizer, resulting in \emph{Saliency-Aware Regularized Quantization Calibration} (SARQC). The proposed regularization encourages quantized weights to remain close to the original weights during calibration, leading to improved generalization at inference time. SARQC integrates seamlessly into existing PTQ pipelines and enhances both scale-search-based and Gram-based methods under a unified formulation. Extensive experiments on dense and Mixture-of-Experts LLMs demonstrate consistent improvements in perplexity and zero-shot accuracy, without introducing additional inference overhead.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑