我们能从数据泄露与机器遗忘中学到哪些法律相关的启示？
What can we learn from Data Leakage and Unlearning for Law?
AI总结：
本研究针对大语言模型的训练数据记忆与泄露问题，发现删除易提取数据后会产生新的易泄露数据点，且微调模型还会泄露预训练数据中的个人身份信息，为相关隐私法律政策制定提供了参考。
摘要：
大语言模型（Large Language Models, LLMs）存在隐私隐患，因为它们会记忆训练数据（包括电子邮件、电话号码等个人身份信息（personally identifiable information, PII））并在推理过程中泄露。企业可能会使用自身的领域定制数据训练LLM，这些数据中也可能包含用户的PII。为了遵守“被遗忘权”等隐私法规，最易被提取的用户数据点可能会被删除。我们研究发现，一旦最易受攻击的数据点被删除，又会有一组新的数据点变得容易被提取。截至目前，学界对微调模型的记忆机制研究关注较少。在本研究中我们还发现，微调模型不仅会泄露自身的训练数据，还会泄露预训练阶段记忆的预训练数据（及其中的PII）。机器遗忘后新数据点变得易被提取、以及预训练数据通过微调模型发生泄露这两个特性，会给使用LLM提供服务的企业带来严重的隐私与法律问题。我们希望本研究能引发人工智能与法律领域的跨学科讨论，探讨制定相关政策应对这些问题的必要性。
英文摘要：
Large Language Models (LLMs) have a privacy concern because they memorize training data (including personally identifiable information (PII) like emails and phone numbers) and leak it during inference. A company can train an LLM on its domain-customized data which can potentially also include their users' PII. In order to comply with privacy laws such as the "right to be forgotten", the data points of users that are most vulnerable to extraction could be deleted. We find that once the most vulnerable points are deleted, a new set of points become vulnerable to extraction. So far, little attention has been given to understanding memorization for fine-tuned models. In this work, we also show that not only do fine-tuned models leak their training data but they also leak the pre-training data (and PII) memorized during the pre-training phase. The property of new data points becoming vulnerable to extraction after unlearning and leakage of pre-training data through fine-tuned models can pose significant privacy and legal concerns for companies that use LLMs to offer services. We hope this work will start an interdisciplinary discussion within AI and law communities regarding the need for policies to tackle these issues.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
