用于低比特大语言模型权重修复的激活加权种子残差编码
Activation-Weighted Seeded Residual Coding for Low-Bit LLM Weight Repair
- Huawei Boole RC（华为布尔研究中心）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
针对低比特大语言模型量化的误差问题，提出AWSRC编解码器修复权重，在Qwen2.5-3B-Instruct上可显著缩小与BF16模型的质量差距，且侧载体积小、性能优于同类编解码器。
AI中文摘要：
低比特权重量化可节省存储，但会产生降低语言模型质量的误差。我们提出激活加权种子残差编码（AWSRC），这是一种用于现有量化主干的紧凑修复编解码器。给定重构权重W₀，AWSRC使用确定性种子生成的基对残差W-W₀进行编码。侧载存储种子选择器、低比特系数和缩放因子，而非显式码本。激活统计优先考虑影响层输出的误差。在Qwen2.5-3B-Instruct上，向INT4 RTN主干添加每权重0.162范围比特，可消除与BF16匹配的困惑度（PPL）、KL散度和准确率差距的88.2%、78.9%和71.3%。修复匹配的强低比特主干也可改善所有测量的质量指标。在匹配的49.25 MB侧载（约为BF16模型权重载荷的0.8%）下，AWSRC在稀疏、低秩和矢量量化编解码器中实现了最佳困惑度和平均任务准确率。
英文摘要：
Low-bit weight quantization saves storage but leaves errors that degrade LLM quality. We introduce activation-weighted seeded residual coding (AWSRC), a compact repair codec for an existing quantization backbone. Given a reconstructed weight $W_0$, AWSRC encodes the residual $W-W_0$ using deterministic seed-generated bases. The sidecar stores seed selectors, low-bit coefficients, and scales rather than an explicit codebook. Two variants combine activation weighting with per-module byte quotas ($\mathrm{AWSRC\text{-}U}$), or blended activation/Fisher weighting with globally ranked progressive prefixes ($\mathrm{AWSRC\text{-}P}_{F}$) that support multiple byte budgets without refitting. On Qwen2.5-3B-Instruct, adding $0.162$ scope-bits/weight to an RTN-INT4 baseline closes $88.2\%$, $78.9\%$, and $71.3\%$ of the PPL, KL, and 11-task mean-accuracy gaps to BF16, respectively. AWSRC achieves the highest mean downstream accuracy in byte-matched residual-codec ablations and improves all metrics across model families with up to 32B parameters.
