arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.00800cs.LGcs.AI
收藏

MagR:用于增强训练后量化的权重幅度缩减

MagR: Weight Magnitude Reduction for Enhancing Post-Training Quantization

University at Albany, SUNY(纽约州立大学奥尔巴尼分校);IBM T. J. Watson Research Center(IBM T. J. 沃森研究中心)

Aozhong Zhang, Naigang Wang, Yanxia Deng, Xin Li, Zi Yang, Penghang Yin

AI总结:

本文提出了一种名为 MagR 的基于优化的权重幅度缩减预处理技术,通过 $\ell_\infty$-正则化平滑异常值且不引入推理开销,在 Llama 系列模型的训练后量化中实现了最先进的性能。

摘要:

在本文中,我们提出了一种简单的基于优化的预处理技术,称为 Weight Magnitude Reduction (MagR),以提升训练后量化的性能。对于每个线性层,我们通过求解一个 $\ell_\infty$-正则化优化问题来调整预训练的浮点权重。该过程极大地减小了权重的最大幅度并平滑了异常值,同时保留了该层的输出。预处理后的权重更集中于零附近,这有助于后续的量化过程。为了实现 MagR,我们采用了一种高效的近端梯度下降算法来处理 $\ell_\infty$-正则化。与涉及线性变换和后续后处理步骤的现有预处理方法不同,这些方法在推理时可能引入显著的开销,而 MagR 作为一种非线性变换,无需任何额外的后处理。这确保了 MagR 在推理期间不引入任何开销。我们的实验表明,MagR 在 Llama 系列模型上实现了最先进的性能。例如,在 LLaMA2-70B 模型上,对于 per-channel INT2 权重量化,我们在不产生任何推理开销的情况下实现了 5.95 的 Wikitext2 困惑度。

英文摘要:

In this paper, we present a simple optimization-based preprocessing technique called Weight Magnitude Reduction (MagR) to improve the performance of post-training quantization. For each linear layer, we adjust the pre-trained floating-point weights by solving an $\ell_\infty$-regularized optimization problem. This process greatly diminishes the maximum magnitude of the weights and smooths out outliers, while preserving the layer's output. The preprocessed weights are centered more towards zero, which facilitates the subsequent quantization process. To implement MagR, we address the $\ell_\infty$-regularization by employing an efficient proximal gradient descent algorithm. Unlike existing preprocessing methods that involve linear transformations and subsequent post-processing steps, which can introduce significant overhead at inference time, MagR functions as a non-linear transformation, eliminating the need for any additional post-processing. This ensures that MagR introduces no overhead whatsoever during inference. Our experiments demonstrate that MagR achieves state-of-the-art performance on the Llama family of models. For example, we achieve a Wikitext2 perplexity of 5.95 on the LLaMA2-70B model for per-channel INT2 weight quantization without incurring any inference overhead.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑