基于差分隐私的合成文本生成:一个简单实用的方法
Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe
AI总结:
本文提出仅用差分隐私微调预训练生成式语言模型的简单方案,在基准与客户数据上生成效用接近非隐私方法且隐私保护强的合成文本。
摘要:
由于机器学习模型倾向于记忆敏感训练数据,数据驱动产品中的隐私问题日益受到关注。为此类数据生成具有正式隐私保证(如差分隐私,DP)的合成版本,为缓解这些隐私担忧提供了一条有前景的途径,但以往沿此方向的方法通常无法生成高质量的合成数据。在本研究中,我们表明文本领域中一个简单实用的方法是有效的:只需使用DP微调预训练的生成式语言模型,就能使模型生成具有强隐私保护的有用合成文本。通过在基准数据和私有客户数据上进行的大量实证分析,我们证明该方法生成的合成文本在效用上可与非隐私对应方法相竞争,同时为潜在的隐私泄露提供强有力的保护。
英文摘要:
Privacy concerns have attracted increasing attention in data-driven products due to the tendency of machine learning models to memorize sensitive training data. Generating synthetic versions of such data with a formal privacy guarantee, such as differential privacy (DP), provides a promising path to mitigating these privacy concerns, but previous approaches in this direction have typically failed to produce synthetic data of high quality. In this work, we show that a simple and practical recipe in the text domain is effective: simply fine-tuning a pretrained generative language model with DP enables the model to generate useful synthetic text with strong privacy protection. Through extensive empirical analyses on both benchmark and private customer data, we demonstrate that our method produces synthetic text that is competitive in terms of utility with its non-private counterpart, meanwhile providing strong protection against potential privacy leakages.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。