arXiv 2310.154692024-07-26cs.CRcs.AIcs.CLcs.LG

Janus接口：大型语言模型中的微调如何放大隐私风险

The Janus Interface: How Fine-Tuning in Large Language Models Amplifies the Privacy Risks

Indiana University Bloomington（印第安纳大学伯明顿分校）；JD Cloud（京东云）；Stanford（斯坦福大学）；CISPA（德国莱布尼茨安全与隐私研究中心）

Xiaoyi Chen, Siyuan Tang, Rui Zhu, Shijun Yan, Lei Jin, Zihao Wang, Liya Su, Zhikun Zhang, XiaoFeng Wang, Haixu Tang

备注：: This work has been accepted by CCS 2024
发表信息：: ACM CCS 2024

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出名为Janus的新型攻击方法，利用大型语言模型的微调接口恢复预训练数据中被遗忘的个人身份信息，其隐私风险放大效果超基线10倍，显著优于现有攻击，并验证了主流微调API的易受攻击性。

摘要：

大型语言模型（LLMs）的快速发展引发了公众对其庞大训练数据集中个人身份信息（PII）隐私泄露的担忧。近期研究表明，攻击者可以通过精心设计的提示词从LLMs的训练数据中提取高度敏感的隐私数据。然而，这些攻击受限于模型在预训练阶段产生的幻觉倾向和灾难性遗忘（CF），导致泄露的PIIs真实性大打折扣。在我们的研究中，我们提出了一种名为Janus的新型攻击方法，该方法利用微调接口从LLMs的预训练数据中恢复被遗忘的PIIs。我们形式化了LLMs中的隐私泄露问题，并通过在开源语言模型上的实证分析解释了为何被遗忘的PIIs能够被恢复。基于这些洞察，我们在开源语言模型以及两个最新的LLMs（即GPT-3.5-Turbo和LLaMA-2-7b）上评估了Janus的性能。实验结果表明，与基线相比，Janus将隐私风险放大了10倍以上，并且显著优于包括前缀攻击和上下文学习（ICL）在内的最先进隐私提取攻击。此外，我们的分析验证了OpenAI和Azure AI Studio提供的现有微调API容易受到我们的Janus攻击，使攻击者能够以低成本实施此类攻击。

英文摘要：

The rapid advancements of large language models (LLMs) have raised public concerns about the privacy leakage of personally identifiable information (PII) within their extensive training datasets. Recent studies have demonstrated that an adversary could extract highly sensitive privacy data from the training data of LLMs with carefully designed prompts. However, these attacks suffer from the model's tendency to hallucinate and catastrophic forgetting (CF) in the pre-training stage, rendering the veracity of divulged PIIs negligible. In our research, we propose a novel attack, Janus, which exploits the fine-tuning interface to recover forgotten PIIs from the pre-training data in LLMs. We formalize the privacy leakage problem in LLMs and explain why forgotten PIIs can be recovered through empirical analysis on open-source language models. Based upon these insights, we evaluate the performance of Janus on both open-source language models and two latest LLMs, i.e., GPT-3.5-Turbo and LLaMA-2-7b. Our experiment results show that Janus amplifies the privacy risks by over 10 times in comparison with the baseline and significantly outperforms the state-of-the-art privacy extraction attacks including prefix attacks and in-context learning (ICL). Furthermore, our analysis validates that existing fine-tuning APIs provided by OpenAI and Azure AI Studio are susceptible to our Janus attack, allowing an adversary to conduct such an attack at a low cost.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。