结构化变换用于语言模型的低开销量化

中文摘要 AI 辅助

我们重新审视了基于Kashin分解的大语言模型权重量化方法，并提出了一种改进算法，该算法具有更强的收敛性质和结构化、高效的正交变换。该方法保留了每个权重分解为两个分量的核心因子分解——一个具有有界无穷范数，另一个在正交变换后具有有界无穷范数——但将稠密随机正交矩阵替换为符号随机化的离散余弦变换（DCT），将每次迭代的成本从$\u039f(N^2)$降低到$\u039f(N \u005c log N)$。所提出的具有交替更新的贪心算法保证了每个因子稳定2比特聚类所需的四峰分布，并允许聚类中心的闭式初始化，消除了先前工作中多重启k-means的瓶颈。与OPTQ风格的顺序误差补偿和QuIP风格的不相干预处理相结合，所得到的JAX流水线在OPT、Llama-2和Pythia上以每通道4比特进行量化时，与OPTQ、QuIP、QuIP-RG以及QuIP#的无微调和无向量量化变体相比具有竞争力，并具有有利的墙钟时间扩展。有界$\u2113_\u221e$因子分解也特别稳健：在QuIP变体发散到四位困惑度（Pythia-6.9B）或在LDL回代中因NaN中止（Mistral-7B）的压力配置下，Kashin-DCT保持数值稳定，并接近FP16基线。在推理时，每个权重分解为每通道两个2比特因子码，这些因子码在结构上适用于原生2比特硬件。

英文摘要

We revisit Kashin-decomposition-based weight quantization for large language models and propose an improved algorithm with stronger convergence properties and structured, efficient orthogonal transforms. The method retains the core factorization of each weight into two components -- one with bounded infinity norm and the other with bounded infinity norm after an orthogonal transformation -- but replaces the dense random orthogonal matrix with a sign-randomized Discrete Cosine Transform (DCT), reducing the per-iteration cost from $\mathcal{O}(N^2)$ to $\mathcal{O}(N \log N)$. The proposed greedy algorithm with alternating updates guarantees the four-peak distribution required for stable 2-bit clustering of each factor and admits closed-form initialization of cluster centers, removing the multi-restart k-means bottleneck of prior work. Composed with OPTQ-style sequential error compensation and QuIP-style incoherence preprocessing, the resulting JAX pipeline is competitive with OPTQ, QuIP, QuIP-RG and a fine-tuning- and vector-quantization-free variant of QuIP# at 4-bit per channel on OPT, Llama-2 and Pythia, with favorable wall-clock scaling. The bounded-$\ell_\infty$ factorization is also notably robust: on stress configurations where QuIP variants diverge to four-digit perplexity (Pythia-6.9B) or abort with NaNs in LDL back-substitution (Mistral-7B), Kashin-DCT remains numerically stable and stays close to FP16 baseline. At inference time, each weight decomposes into two 2-bit factor codes per channel that are structurally suited to native-2-bit hardware.

发表机构

• Institute of Numerical Mathematics（数值数学研究所）
• Steklov Mathematical Institute（斯捷克洛夫数学研究所）

机构由 AI 辅助整理，请以论文原文为准。