AI 中文总结
研究提出KronQ框架，通过引入梯度协方差挑战现有二阶PTQ方法假设。利用克罗内克分解海森矩阵近似，在两个层面改进量化：双向非相干处理减少权重幅度方差，导出新敏感度度量用于层间混合精度分配，在特定量化任务中表现优于GPTQ等。
AI 中文摘要
训练后量化（PTQ）是一种广泛采用的无需重新训练即可压缩大语言模型（LLMs）的技术。现有的二阶PTQ方法，包括GPTQ，仅从输入激活统计构建量化目标，有效假设所有输出通道对逐层重建目标贡献均等。我们提出KronQ，一个通过将梯度协方差引入量化管道来挑战此假设的PTQ框架。在克罗内克分解海森矩阵近似下，量化损失联合依赖于激活和梯度协方差，KronQ在两个互补层面利用此点。（1）KronQ引入双向非相干处理，使用梯度协方差将现有输入侧随机旋转扩展到输出维度，减少输入和输出维度上的权重幅度方差。（2）KronQ由梯度和激活海森矩阵迹驱动，导出用于层间混合精度分配的新敏感度度量。值得注意的是，在对LLaMA - 3 - 70B进行2位仅权重量化时，GPTQ和GPTAQ发散或产生退化量化（WikiText - 2上>2000困惑度），而KronQ实现了7.93困惑度。
英文摘要
Post-training quantization (PTQ) is a widely adopted technique for compressing large language models (LLMs) without retraining. Most existing second-order PTQ methods, including GPTQ, construct quantization objectives from input activation statistics, effectively assuming that all output channels contribute equally to the layer-wise reconstruction objective. We propose KronQ, a PTQ framework that challenges this assumption by introducing the gradient covariance into the quantization pipeline. Under the Kronecker-factored Hessian approximation, the quantization loss depends jointly on both the activation and gradient covariances, and KronQ exploits this at two complementary levels. (1) KronQ introduces bidirectional incoherence processing, extending the existing input-side random rotation to the output dimension using the gradient covariance, reducing weight magnitude variance across both input and output dimensions. (2) KronQ derives a new sensitivity metric for inter-layer mixed-precision allocation, driven by the gradient and activation Hessian traces. Notably, in the case of 2-bit weight-only quantization on LLaMA-3-70B, while GPTQ and GPTAQ diverge or produce degenerate quantizations (>2000 perplexity on WikiText-2), \KronQ{} achieves 7.93 perplexity.
CommentsCOLM 2026
