AI 中文总结
研究语言模型量化中逐字提取情况，用Pythia模型及公共序列集，在多精度级别和模型大小下跟踪，发现量化是有选择性的遗忘器，但不足以成隐私保护手段，得出压缩非去记忆训练数据方法，从业者应关注提取的结论。
AI 中文摘要
语言模型在部署前几乎总是被量化，越来越多的工作探讨量化是否也降低了它们的隐私风险。以往工作几乎完全通过成员推理来衡量隐私。但人们真正担心的是模型逐字重现训练数据，我们对此直接进行测量。使用Pythia模型及其已知记忆的公共序列集，我们在从全精度到4比特的五个精度级别以及三种模型大小上跟踪逐字提取，同时测量每个点的一般能力（困惑度）。我们发现：量化是一种有选择性的遗忘器，在我们尝试的每个精度和模型大小下，逐字记忆比能力下降得更快，这在两种无关的量化算法和两个评估语料库下都成立；但这种选择性不足以使量化成为一种隐私保护手段。在我们研究的最大模型中，4比特量化仍能重现大部分记忆序列，同时仅放弃百分之几的能力，且量化后幸存的记忆数据比例随模型大小增加。我们得出结论，压缩不应被视为去除记忆训练数据的方法，从业者应关注提取而非成员推理。所有代码、采样评估数据和每个配置的结果都已发布。
英文摘要
Language models are almost always quantized before they are deployed, and a growing line of work asks whether quantization also lowers their privacy risk. That work measures privacy almost entirely with membership inference. We think this is the wrong thing to measure for the risk that most people actually worry about, namely a model reproducing its training data word for word, and we measure that directly. Using the Pythia models and the public set of sequences each of them is known to have memorized, we track verbatim extraction across five precision levels, from full precision down to four bits, and across three model sizes, while measuring general capability (perplexity) at every point. We find two things. Quantization is a selective forgetter: verbatim memorization falls off faster than capability at every precision and every model size we tried, and this holds under two unrelated quantization algorithms and two evaluation corpora. But the selectivity is not enough to make quantization a privacy defense, which cuts against the optimistic reading of earlier membership-inference results. At the largest model we study, four-bit quantization still reproduces most of the memorized sequences while giving up only a few percent of capability, and the fraction of memorized data that survives quantization grows with model size. We conclude that compression should not be treated as a way to remove memorized training data, and that extraction, not membership inference, is the number practitioners should be watching. All code, sampled evaluation data, and per-configuration results are released.
Comments8 pages, 3 figures. Code: https://github.com/AkshaySasi/bits-and-memories. Data and results: https://huggingface.co/datasets/AkshaySasi/bits-and-memories
