发表机构
Black Sesame Technologies（黑芝麻科技）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
提出 CurveTQ，一种无旋转的 LLM 权重量化方法，通过利用 Hessian 曲率权重进行网格搜索，在两比特下提升下游准确率并加速解码。
AI 中文摘要
大型语言模型的最佳两比特权重量化器（如 QTIP 和 Proteus）通过随机正交变换旋转每个权重矩阵，该变换必须在每次解码步骤中撤销，然后使用网格或格码在欧几里得搜索下进行编码；层 Hessian 矩阵仅通过编码块之间的误差反馈进入。我们表明这留下了部分 Hessian 未被使用。误差反馈将损失转化为每个坐标舍入误差的加权和，其权重是 Hessian 的 LDL 分解的对角线，现有量化器计算了这些权重但从未读取。我们将这些权重放入 Viterbi 分支度量中，因此搜索遵循每个编码块内的曲率。这也解释了旋转：它移除了块内变化，因此在原始基中加权与旋转是替代方案。在三个模型上，加权原始搜索与全维度随机 Hadamard 变换在下游准确率上相差约一个点，而旋转后加权几乎没有增益。围绕此搜索，我们构建了 CurveTQ，一种无旋转的网格编解码器，它通过因子化尺度场和闭式分位数表处理权重的幅度和边际形状，并为每个编码块存储起始状态，以便网格适应误差反馈带入的残差。在两比特下，CurveTQ 在三个 4-8B Instruct 模型上的平均下游准确率比 QTIP 和 Proteus 高 1-3 个百分点，即使在两者都使用我们的起始状态后也是如此，而起始状态本身可将任一基线提升 1-3 个百分点。它还在 35B 混合专家模型上领先，据我们所知，这是此类模型上首个网格编码结果。由于无需撤销旋转，我们的解码器在所有测试的批大小和位宽下都是三者中最快的。
英文摘要
The best two-bit weight quantizers for large language models, such as QTIP and Proteus, rotate each weight matrix by a random orthogonal transform, which must be undone at every decoding step, then encode it with a trellis or lattice code under a Euclidean search; the layer Hessian enters only through error feedback between coding blocks. We show that this leaves part of the Hessian unused. Error feedback turns the loss into a weighted sum of per-coordinate rounding errors whose weights, the diagonal of the Hessian's LDL factorization, existing quantizers compute but never read. We put these weights into the Viterbi branch metric, so the search follows the curvature within each coding block. This also explains the rotation: it removes this within-block variation, so weighting in the native basis and rotating are substitutes. On three models the weighted native search matches a full-dimension randomized Hadamard to within about one point of downstream accuracy, and weighting after the rotation gains little. Around this search we build CurveTQ, a trellis codec with no rotation, which handles the weights' amplitude and marginal shape with a factored scale field and a closed-form quantile table, and stores a start state per coding block so the trellis can adapt to the residual that error feedback carries into it. At two bits CurveTQ is 1-3 points higher in mean downstream accuracy than QTIP and Proteus on three 4-8B Instruct models, even after both are given our start state, which alone lifts either baseline by 1-3 points. It also leads on a 35B mixture of experts, to our knowledge the first trellis-coded result on such a model. With no rotation to undo, our decoder is the fastest of the three at all tested batch sizes and bit widths.
