压缩与遗忘：bitsandbytes量化放大了大语言模型中的主动干扰
Compress and Forget: bitsandbytes Quantization Amplifies Proactive Interference in LLMs
- Shahid Beheshti University（沙希德·贝赫什提大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
该研究发现bitsandbytes 4位量化会放大大语言模型的主动干扰，降低高干扰场景下的检索准确率，其效应源于量化后的Transformer主干，对依赖长语义密集上下文的应用有额外成本。
AI中文摘要：
主动干扰（Proactive Interference, PI）是大语言模型中已被证实的一种失效模式，即随着先前覆盖操作的累积，对被反复覆盖值的检索能力会下降，这与人类工作记忆中的经典现象类似。后训练量化（Post-training Quantization, PTQ）目前是开放权重模型的默认部署路径，但其对该失效模式的影响尚未被测试。我们在固定检索任务的前提下，评估了三种精度级别（FP16、INT8、通过bitsandbytes实现的INT4/NF4）在三个架构不同的指令微调模型（Qwen2.5-7B-Instruct、Mistral-7B-Instruct-v0.3、Phi-3.5-mini-instruct）上的表现。INT4量化在所有模型的高干扰场景下均显著降低了准确率（例如Qwen模型从81.0%降至68.3%），经配对McNemar检验（p≤2.6×10⁻⁶）和覆盖所有干扰级别的混合效应回归验证；常被认为安全的INT8在三个模型中的两个也存在较小但真实的性能损失。该效应仅针对语义相似的（词型）干扰项，在数值控制条件下会反转符号，且在机制上与INT4下同键侵入错误的增加相关（从21.5%升至24.6%的试验，p=4.8×10⁻⁷）。后续消融实验表明，该效应源于量化后的Transformer主干而非输出投影层。这些结果表明，即使在整体基准准确率看似基本未受影响的情况下，bitsandbytes 4位量化仍会对依赖长、可更新、语义密集上下文的应用造成额外成本。我们在该httpsURL发布了代码及经分词器验证的词汇表构建方法。
英文摘要：
Proactive interference (PI) is a documented failure mode in large language models in which retrieval of a repeatedly overwritten value degrades as prior overwrites accumulate, mirroring a classical phenomenon in human working memory. Post-training quantization (PTQ) is now the default deployment path for open-weight models, yet its effect on this failure mode has not been tested. We evaluate three precision levels (FP16, INT8, INT4/NF4, via bitsandbytes) across three architecturally distinct instruction-tuned models (Qwen2.5-7B-Instruct, Mistral-7B-Instruct-v0.3, Phi-3.5-mini-instruct), holding the retrieval task fixed. INT4 quantization significantly reduces accuracy under high interference in every model (e.g., from 81.0% to 68.3% for Qwen), confirmed by paired McNemar's tests ($p \le 2.6 \times 10^{-6}$) and a mixed-effects regression spanning all interference levels; INT8, often assumed safe, also carries a smaller but real penalty in two of three models. The effect is specific to semantically similar (word-type) distractors and reverses sign under a numeric control condition, and is mechanistically linked to a rise in same-key intrusion errors under INT4 (from 21.5% to 24.6% of trials, $p = 4.8 \times 10^{-7}$). A follow-up ablation shows the effect originates in the quantized transformer backbone rather than the output projection layer. These results suggest that bitsandbytes 4-bit quantization can impose an additional cost on applications relying on long, updatable, semantically dense contexts, even when aggregate benchmark accuracy appears largely unaffected. We release our code and tokenizer-verified vocabulary construction method at https://github.com/ShayanShahrabi/compress-and-forget
