arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.22347cs.LGcs.AI
收藏

推动后训练量化中块旋转的极限

Pushing the Limits of Block Rotations in Post-Training Quantization

Advanced Micro Devices, Inc. (AMD)(Advanced Micro Devices公司);State University of New York at Buffalo(纽约州立大学布法罗分校);Norwegian University of Science(挪威科学与技术大学)

Sai Sanjeet, Ian Colbert, Pablo Monteagudo-Lago, Giuseppe Franco, Yaman Umuroglu, Nicholas J. Fraser

AI总结:

本文提出PeRQ框架,通过置换和旋转重新分布激活值,以克服块旋转在抑制异常值时的几何限制,显著提升后训练量化精度。

摘要:

最近的后训练量化(PTQ)方法采用块旋转来在舍入前扩散异常值。虽然这减少了在线全向量旋转的开销,但块结构对异常值抑制的影响仍知之甚少。为填补这一空白,我们首次对块Hadamard旋转的异常值抑制进行了系统的非渐近分析。我们的分析表明,异常值抑制从根本上受限于输入向量的几何结构。特别地,在确定性最坏情况下,当旋转前的ℓ1范数质量在块间均匀分布时,旋转后的异常值最小。受这些见解的启发,我们引入了PeRQ(置换、旋转、然后量化),一个在旋转前通过置换重新分布激活质量的PTQ框架。我们提出了一种贪婪质量扩散算法,通过均衡期望的块间ℓ1范数来校准置换。为避免增加推理开销,我们识别了Transformer架构中置换等变区域,在部署前将这些置换合并到模型权重中。实验表明,PeRQ在所有块大小上一致地提高了精度,在将Llama3 1B量化为INT4且块大小为16时,恢复了全向量旋转困惑度的90%,而未经置换时仅为46%。

英文摘要:

Recent post-training quantization (PTQ) methods have adopted block rotations to diffuse outliers prior to rounding. While this reduces the overhead of online full-vector rotations, the effect of block structure on outlier suppression remains poorly understood. To fill this gap, we present the first systematic, non-asymptotic analysis of outlier suppression for block Hadamard rotations. Our analysis reveals that outlier suppression is fundamentally limited by the geometry of the input vector. In particular, in the deterministic worst case, post-rotation outliers are minimized when the pre-rotation $\ell_1$ norm mass is evenly distributed across blocks. Guided by these insights, we introduce PeRQ (Permute, Rotate, then Quantize), a PTQ framework that redistributes activation mass via permutations prior to rotation. We propose a greedy mass diffusion algorithm to calibrate permutations by equalizing the expected blockwise $\ell_1$ norms. To avoid adding inference overhead, we identify permutation-equivariant regions in transformer architectures to merge these permutations into model weights before deployment. Experiments show that PeRQ consistently improves accuracy across all block sizes, recovering up to 90% of the full-vector rotation perplexity when quantizing Llama3 1B to INT4 with block size 16, compared to 46% without permutations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑