并非所有4位量化器都相同：微调小型语言模型中PII泄露的部署时缓解
Not All 4-bit Quantizers Are Equal: Deployment-Time Mitigation of PII Leakage in Fine-Tuned Small Language Models
浏览论文内容
中文总结 AI 辅助
研究发现4位量化方法的选择影响微调小型语言模型的隐私泄露，基于校准的方法（如AWQ）比无校准方法泄露更少，部署时应将其视为隐私决策。
中文摘要 AI 辅助
组织在私有数据上微调小型语言模型，然后将其压缩至4位以实现资源高效的部署。我们表明，压缩方法也会影响隐私。区分这些方法的不是位宽，而是它们是否在少量文本样本（即校准语料库）上调整其舍入方式。在我们的主要模型上，当每个植入记录的自身开头文本被用作提示时，我们测试的两种基于校准的方法——激活感知权重量化（AWQ）和基于梯度的训练后量化（GPTQ）——均未重现任何植入记录，而无校准语料库的GGUF Q4_K_M格式则重现了其中5.3%的记录。跨五个参数量为0.5至70亿的开源模型追踪，AWQ在每个规模及两个模型家族中泄露最少，且在30至70亿参数下几乎没有精度损失。受控实验将差异归因于校准引起的舍入误差，这些误差出现在涉及稀有标记预测的通道中。因此，选择4位方法是一个部署时的隐私决策，而不仅仅是速度和质量问题。
英文摘要
Organizations fine-tune small language models on private data and then compress them to 4 bits for resource-efficient deployment. We show that the compression method also affects privacy. What separates the methods is not the bit width but whether they tune their rounding on a small sample of text, the calibration corpus. On our primary model, when each planted record's own opening text is used as the prompt, the two calibration-based methods we test, Activation-aware Weight Quantization (AWQ) and Gradient-based Post-Training Quantization (GPTQ), each reproduce none of the planted records, while the calibration-corpus-free GGUF Q4_K_M format reproduces 5.3% of them. Tracked across five open models with 0.5-7 billion parameters, AWQ leaks least at every size and in both families, with little accuracy loss at 3-7 billion. Controlled experiments associate the difference with calibration-induced rounding error in channels involved in rare-token prediction. Choosing the 4-bit method is therefore a deployment-time privacy decision, not only a question of speed and quality.
发表机构
- AI Horizon Labs（AI地平线实验室）
- Federal University of Pampa (UNIPAMPA)（潘帕联邦大学（UNIPAMPA））
机构由 AI 辅助整理，请以论文原文为准。
