ShamAN-Q：Shampoo增强的NanoQuant亚1比特大语言模型权重量化

中文摘要 AI 辅助

我们提出了ShamAN-Q，一种亚1比特的后训练量化方法，它扩展了NanoQuant，将其每个对角重建几何替换为可处理的稠密曲率度量，采用了由Shampoo优化器推广的通用范式。对于每个线性权重，ShamAN-Q通过Kullback-Leibler最小化，将Kronecker积拟合到小校准集的经验Fisher信息矩阵上，从而形成基于结果的Mahalanobis重建损失。NanoQuant中的连续ADMM更新变为Sylvester方程的解，而其离散投影和部署格式保持不变。由于曲率是相对于给定权重集合局部的，ShamAN-Q在每层因子分解之前立即重新测量该层的输入曲率统计量，并周期性地刷新部分量化模型上的所有统计量。ShamAN-Q还在相同的总比特数下，将NanoQuant中的均匀秩重新分配到各层。在Qwen3-Base上，ShamAN-Q在约1 bpw下将WikiText-2困惑度从27.56降至22.96（0.6B），从19.21降至16.72（1.7B），从14.29降至13.80（4B），同时在Eleuther LM评估工具上匹配或提高了零样本准确率。在0.6B模型上，ShamAN-Q在约0.8 bpw下的表现与NanoQuant在约1.0 bpw下发布的困惑度相匹配。

英文摘要

We introduce ShamAN-Q, a sub-1-bit post-training quantization method that extends NanoQuant by replacing each its diagonal reconstruction geometry with a tractable dense curvature metric, using a general paradigm popularized by the Shampoo optimizer. For each linear weight, ShamAN-Q fits a Kronecker product to the empirical Fisher information matrix of a small calibration set by Kullback--Leibler minimization, forming a Mahalanobis reconstruction loss from the result. The continuous ADMM updates from NanoQuant become solutions to Sylvester equations, while its discrete projection and deployment format remain unchanged. Because the curvature is local to a given set of weights, ShamAN-Q re-measures the input curvature statistic for each layer immediately before layer factorization, periodically refreshing all statistics on the partially quantized model. ShamAN-Q also redistributes the uniform rank from NanoQuant across layers at the same total number of bits. On Qwen3-Base, ShamAN-Q lowers WikiText-2 perplexity at $\approx$1 bpw from 27.56 to 22.96 (0.6B), 19.21 to 16.72 (1.7B), and 14.29 to 13.80 (4B) while matching or improving zero-shot accuracy on the Eleuther LM Evaluation Harness. On 0.6B, ShamAN-Q at $\approx$0.8 bpw matches the published perplexity of NanoQuant at $\approx$1.0 bpw.

发表机构

• IonQ

机构由 AI 辅助整理，请以论文原文为准。