在微调语言模型中意外记忆敏感信息
Unintended Memorization of Sensitive Information in Fine-Tuned Language Models
AI总结：
研究揭示微调语言模型时意外记忆敏感信息的风险，分析影响因素并评估隐私保护方法的权衡。
摘要：
对大型语言模型（LLMs）进行微调时，使用敏感数据集存在重大风险，即意外记忆和泄露个人身份信息（PII），这可能违反隐私法规并危及个人安全。在本工作中，我们系统地研究了一个关键但尚未被充分探索的漏洞：那些仅出现在模型输入中、不在训练目标中的PII的暴露。利用合成和现实世界的数据集，我们设计了受控的提取探针，以量化意外PII记忆并研究语言、PII频率、任务类型和模型大小等因素如何影响记忆行为。我们进一步对四种隐私保护方法进行了基准测试，包括差分隐私、机器无学习、正则化和偏好对齐，评估它们在隐私和任务性能之间的权衡。我们的结果表明，后训练方法通常能提供更一致的隐私-效用权衡，尽管差分隐私在特定设置中能有效减少泄露，但可能会引入训练不稳定性。这些发现突显了微调LLMs中记忆的持续挑战，并强调了需要稳健、可扩展的隐私保护技术的重要性。
英文摘要：
Fine-tuning Large Language Models (LLMs) on sensitive datasets carries a substantial risk of unintended memorization and leakage of Personally Identifiable Information (PII), which can violate privacy regulations and compromise individual safety. In this work, we systematically investigate a critical and underexplored vulnerability: the exposure of PII that appears only in model inputs, not in training targets. Using both synthetic and real-world datasets, we design controlled extraction probes to quantify unintended PII memorization and study how factors such as language, PII frequency, task type, and model size influence memorization behavior. We further benchmark four privacy-preserving approaches including differential privacy, machine unlearning, regularization, and preference alignment, evaluating their trade-offs between privacy and task performance. Our results show that post-training methods generally provide more consistent privacy-utility trade-offs, while differential privacy achieves strong reduction in leakage in specific settings, although it can introduce training instability. These findings highlight the persistent challenge of memorization in fine-tuned LLMs and emphasize the need for robust, scalable privacy-preserving techniques.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
