DP-MLM：使用掩码语言模型的差分隐私文本改写
DP-MLM: Differentially Private Text Rewriting Using Masked Language Models
AI总结：
本文提出 DP-MLM，利用仅编码器掩码语言模型逐 token 进行差分隐私文本改写，在更低 ε 下实现更好的效用保持与机制可定制性，并公开代码。
摘要：
近来，利用差分隐私进行文本私有化的任务采用了“文本改写”的形式，即通过生成式（大型）语言模型对输入文本进行混淆。尽管这些方法在隐私保护能力方面展现出良好效果，但它们依赖自回归模型，而这类模型缺乏使隐私改写过程具备上下文感知的机制。针对这一问题，我们提出 DP-MLM，这是一种用于差分隐私文本改写的新方法，其核心是利用掩码语言模型（MLM），以语义相似且经过混淆的方式改写文本。我们通过一种简单的上下文化技术实现这一点：逐 token 改写文本。我们发现，与以往依赖更大规模解码器模型的方法相比，使用仅编码器 MLM 能在更低的 ε 水平下更好地保留效用。此外，与生成式方法相比，MLM 还允许对改写机制进行更强的定制。我们将 DP-MLM 的代码公开并提供复用，地址为 https://github.com/sjmeis/DPMLM 。
英文摘要：
The task of text privatization using Differential Privacy has recently taken the form of $\textit{text rewriting}$, in which an input text is obfuscated via the use of generative (large) language models. While these methods have shown promising results in the ability to preserve privacy, these methods rely on autoregressive models which lack a mechanism to contextualize the private rewriting process. In response to this, we propose $\textbf{DP-MLM}$, a new method for differentially private text rewriting based on leveraging masked language models (MLMs) to rewrite text in a semantically similar $\textit{and}$ obfuscated manner. We accomplish this with a simple contextualization technique, whereby we rewrite a text one token at a time. We find that utilizing encoder-only MLMs provides better utility preservation at lower $\varepsilon$ levels, as compared to previous methods relying on larger models with a decoder. In addition, MLMs allow for greater customization of the rewriting mechanism, as opposed to generative approaches. We make the code for $\textbf{DP-MLM}$ public and reusable, found at https://github.com/sjmeis/DPMLM .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
