AI 中文总结
研究提出ExTernD方法，将大语言模型权重矩阵因式分解，扩展内部秩纠正量化误差，残差随秩单调递减可逼近bf16精度。该方法内存、计算和因子稀疏性可连续调整，在多个模型上匹配或接近Q4_K/Q5_K精度。
AI 中文摘要
我们引入ExTernD（扩展秩三元分解），一种将每个大语言模型权重矩阵\(A \in \mathbb{R}^{m \times n}\)进行训练后因式分解为\(A \approx B \mathrm{diag}(D) C\)的方法，其中三元因子\(B \in \{-1,0,+1\}^{m \times k}\)，\(C \in \{-1,0,+1\}^{k \times n}\)以及实尺度向量\(D \in \mathbb{R}^k\)。内部秩\(k = \mu \min(m,n)\)被有意扩展到满秩之外（\(\mu > 1\)），使得超出满秩的分量纠正早期分量的量化误差。我们证明残差随\(k\)单调递减且可被驱动到任何\(\varepsilon > 0\)以下：ExTernD能任意接近bf16精度，这是固定平面数的三元方案无法做到的。内存和计算随\(\mu\)连续缩放，因子稀疏性随阈值\(\tau\)连续变化，因此能精确达到精度目标而非舍入到下一位宽。ExTernD在Gemma - 4 - E2B和Qwen3.5 - 4B上，每矩阵精度在5.2 - 5.5有效比特/权重（重要性加权时为5.1 - 5.5）上匹配Q4_K，并且在\(\mu = 3\)时对完整的Qwen3.5 - 4B进行转换，在wikitext - 2上达到10.10的困惑度，而bf16为9.78（+3.2%），使其在约5.7有效比特/权重时接近Q4_K/Q5_K精度带。
英文摘要
We introduce ExTernD (Expanded-rank Ternary Decomposition), a post-training factorization of each LLM weight matrix $A \in \mathbb{R}^{m \times n}$ into $A \approx B \mathrm{diag}(D) C$ with ternary factors $B \in \{-1,0,+1\}^{m \times k}$, $C \in \{-1,0,+1\}^{k \times n}$ and a real scale vector $D \in \mathbb{R}^k$. The inner rank $k = μ\min(m,n)$ is deliberately expanded beyond full rank ($μ> 1$), so that components past full rank correct the quantization error of earlier ones. We prove the residual decreases monotonically in $k$ and can be driven below any $\varepsilon > 0$: ExTernD approaches bf16 accuracy arbitrarily closely, which no ternary scheme with a fixed plane count can do. Memory and compute scale continuously with $μ$, and factor sparsity continuously with a threshold $τ$, so an accuracy target is hit exactly rather than rounded to the next bit-width. ExTernD matches Q4_K's per-matrix accuracy at 5.2-5.5 effective bpw (5.1-5.5 with importance weighting) on Gemma-4-E2B and Qwen3.5-4B, and a full Qwen3.5-4B conversion at $μ= 3$ reaches 10.10 wikitext-2 perplexity against 9.78 for bf16 (+3.2%), placing it near the Q4_K/Q5_K accuracy band at ~5.7 effective bpw.
