一种基于搭配的方法，用于应对词级度量差分隐私中的挑战
A Collocation-based Method for Addressing Challenges in Word-level Metric Differential Privacy
AI总结：
本文提出在词级与句子级之间的搭配层面扰动n-gram，通过构建融合一元词与多元搭配的嵌入模型，提升词级度量差分隐私输出的语义连贯性和长度灵活性，并以效用与隐私实验证明超越词级令牌化策略的必要性。
摘要：
差分隐私（DP）在自然语言处理中的应用必须区分所提出机制运作的句法层级，通常表现为词级或文档级私有化。近来，已提出若干词级度量差分隐私方法，它们依赖这种广义DP概念在词嵌入空间中运作。然而，这些方法往往无法生成语义连贯的文本输出，并且它们在句子级或文档级的应用只能通过对词扰动进行基本组合来实现。在本工作中，我们通过在词级与句子级之间、即搭配层面进行运作，力求应对这些挑战。通过扰动n-gram而非单个词，我们设计了一种方法，使组合后的私有化输出具有更高的语义连贯性和可变长度。具体做法是构建一个基于高频词组的嵌入模型，其中一元词与二元和三元搭配共存。我们在效用和隐私测试中评估了该方法，结果为超越词级的令牌化策略提供了明确依据。
英文摘要：
Applications of Differential Privacy (DP) in NLP must distinguish between the syntactic level on which a proposed mechanism operates, often taking the form of $\textit{word-level}$ or $\textit{document-level}$ privatization. Recently, several word-level $\textit{Metric}$ Differential Privacy approaches have been proposed, which rely on this generalized DP notion for operating in word embedding spaces. These approaches, however, often fail to produce semantically coherent textual outputs, and their application at the sentence- or document-level is only possible by a basic composition of word perturbations. In this work, we strive to address these challenges by operating $\textit{between}$ the word and sentence levels, namely with $\textit{collocations}$. By perturbing n-grams rather than single words, we devise a method where composed privatized outputs have higher semantic coherence and variable length. This is accomplished by constructing an embedding model based on frequently occurring word groups, in which unigram words co-exist with bi- and trigram collocations. We evaluate our method in utility and privacy tests, which make a clear case for tokenization strategies beyond the word level.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
