arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2101.05405cs.CRcs.CLcs.LG
收藏

语言模型训练数据泄漏分析

Training Data Leakage Analysis in Language Models

Microsoft Research(微软研究院);Microsoft Corporation(微软公司)

Huseyin A. Inan, Osman Ramadan, Lukas Wutschitz, Daniel Jones, Victor Rühle, James Withers, Robert Sim

摘要:

基于神经网络的语言模型的最新进展导致了这种模型的成功部署,改善了各种应用中的用户体验。事实证明,语言模型的强大性能伴随着记忆稀有训练样本的能力,如果模型是在机密用户内容上训练的,这会带来严重的隐私威胁。在这项工作中,我们介绍了一种方法,调查识别用户内容的训练数据,可以在一个强大的和现实的威胁模型下泄漏。我们提出了两个指标来量化用户级的数据泄漏测量模型的能力,产生独特的句子片段内训练数据。我们的指标还可以在隐私方面比较在相同数据上训练的不同模型。我们通过对RNN和基于Transformer的模型进行广泛的数值研究来证明我们的方法。我们进一步说明了如何利用所提出的指标来调查缓解措施的有效性,如差异化私有培训或API强化。

英文摘要:

Recent advances in neural network based language models lead to successful deployments of such models, improving user experience in various applications. It has been demonstrated that strong performance of language models comes along with the ability to memorize rare training samples, which poses serious privacy threats in case the model is trained on confidential user content. In this work, we introduce a methodology that investigates identifying the user content in the training data that could be leaked under a strong and realistic threat model. We propose two metrics to quantify user-level data leakage by measuring a model's ability to produce unique sentence fragments within training data. Our metrics further enable comparing different models trained on the same data in terms of privacy. We demonstrate our approach through extensive numerical studies on both RNN and Transformer based models. We further illustrate how the proposed metrics can be utilized to investigate the efficacy of mitigations like differentially private training or API hardening.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑