PreCurious：无辜的预训练语言模型如何沦为隐私陷阱
PreCurious: How Innocent Pre-Trained Language Models Turn into Privacy Traps
AI总结：
本研究提出PreCurious框架，揭示攻击者通过发布恶意预训练模型突破参数高效、差分隐私等防御，窃取微调数据集隐私的新型攻击面，警示用户警惕未知来源预训练模型等相关风险。
摘要：
预训练-微调范式已被证明具备有效性，成为为各类任务定制语言模型的标准方法。目前，社区平台可让用户轻松获取各类预训练模型，任何人无需经过严格验证流程即可发布模型。然而，若经过精心设计，发布的预训练模型可能成为微调数据集的隐私陷阱。在本研究中，我们提出PreCurious框架，以揭示攻击者发布预训练模型并获得最终微调模型黑盒访问权限的新型攻击面。PreCurious旨在升级微调数据集上成员推理与数据提取两类通用隐私风险。PreCurious的核心思路是操控预训练模型的记忆阶段，并通过看似合理的配置引导微调过程。尽管实验与理论证据表明，参数高效微调技术和差分隐私微调技术可防御针对微调模型的隐私攻击，但与在良性预训练模型上微调相比，PreCurious证明了以隐蔽方式突破这种防护的可能性。虽然差分隐私（DP）对成员推理攻击有一定缓解作用，但通过进一步利用经过清洗的数据集，PreCurious证明即使在严格隐私预算（例如ε=0.05）的差分隐私微调下，定向数据提取仍存在潜在漏洞。因此，PreCurious向用户发出警告：从未知来源下载预训练模型、仅依赖教程或常识性防御措施，以及即便经过彻底清洗仍发布清洗后的数据集，都存在潜在风险。
英文摘要：
The pre-training and fine-tuning paradigm has demonstrated its effectiveness and has become the standard approach for tailoring language models to various tasks. Currently, community-based platforms offer easy access to various pre-trained models, as anyone can publish without strict validation processes. However, a released pre-trained model can be a privacy trap for fine-tuning datasets if it is carefully designed. In this work, we propose PreCurious framework to reveal the new attack surface where the attacker releases the pre-trained model and gets a black-box access to the final fine-tuned model. PreCurious aims to escalate the general privacy risk of both membership inference and data extraction on the fine-tuning dataset. The key intuition behind PreCurious is to manipulate the memorization stage of the pre-trained model and guide fine-tuning with a seemingly legitimate configuration. While empirical and theoretical evidence suggests that parameter-efficient and differentially private fine-tuning techniques can defend against privacy attacks on a fine-tuned model, PreCurious demonstrates the possibility of breaking up this invulnerability in a stealthy manner compared to fine-tuning on a benign pre-trained model. While DP provides some mitigation for membership inference attack, by further leveraging a sanitized dataset, PreCurious demonstrates potential vulnerabilities for targeted data extraction even under differentially private tuning with a strict privacy budget e.g. $ε=0.05$. Thus, PreCurious raises warnings for users on the potential risks of downloading pre-trained models from unknown sources, relying solely on tutorials or common-sense defenses, and releasing sanitized datasets even after perfect scrubbing.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
