在语言模型训练中添加或移除个人信息带来的隐私连锁反应
Privacy Ripple Effects from Adding or Removing Personal Information in Language Model Training
AI总结：
本文研究了在LLM训练中添加或移除PII对隐私的连锁影响，发现PII记忆是动态属性，并揭示了辅助记忆、添加PII增强其他记忆、移除PII导致新记忆三种现象，建议模型创建者防范此类一阶与二阶隐私风险。
摘要：
由于个人身份信息（PII）的敏感性，其所有者可能有权控制其是否被纳入大型语言模型（LLM）训练或要求将其移除。除此之外，由于数据集整理技术的演进、重新训练时重新抓取数据，或被纳入新的下游微调阶段，PII也可能在训练数据集中被添加或移除。我们发现，PII被记忆的数量和难易程度是模型的一种动态属性，它在整个训练流程中不断演变，并依赖于经常被更改的设计选择。我们刻画了三种此类新现象：（1）在训练后期出现的相似PII能够诱发对早期已见序列的记忆，我们称之为辅助记忆，这是一个重要因素（在我们的设置中，最高达1/3）；（2）添加PII可以显著增加对其他PII的记忆（在我们的设置中，高达约7.5倍）；以及（3）移除PII可能导致其他PII被记忆。模型创建者在训练模型时应考虑这些一阶和二阶隐私风险，以避免新PII被重新生成的风险。
英文摘要：
Due to the sensitive nature of personally identifiable information (PII), its owners may have the authority to control its inclusion or request its removal from large-language model (LLM) training. Beyond this, PII may be added or removed from training datasets due to evolving dataset curation techniques, because they were newly scraped for retraining, or because they were included in a new downstream fine-tuning stage. We find that the amount and ease of PII memorization is a dynamic property of a model that evolves throughout training pipelines and depends on commonly altered design choices. We characterize three such novel phenomena: (1) similar-appearing PII seen later in training can elicit memorization of earlier-seen sequences in what we call assisted memorization, and this is a significant factor (in our settings, up to 1/3); (2) adding PII can increase memorization of other PII significantly (in our settings, as much as $\approx\!7.5\times$); and (3) removing PII can lead to other PII being memorized. Model creators should consider these first- and second-order privacy risks when training models to avoid the risk of new PII regurgitation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
