通过自然文本清理实现文本分析的差分隐私
Differential Privacy for Text Analytics via Natural Text Sanitization
摘要：
文本传达复杂的知识。然而，文本也传递敏感信息。尽管具有差分隐私（DP）的通用语言模型和领域特定机制取得了成功，但是现有的文本净化机制仍然提供低效用，因为受到高维文本表示的诅咒。利用净化文本进行下游分析的伴随问题也未得到充分探讨。本文采用直接的方法来进行文本清理。我们的见解是通过我们新的本地DP概念来考虑敏感性和相似性。经过清理的文本还有助于我们的清理感知预训练和微调，从而能够通过具有良好实用性的BERT语言模型进行隐私保护的自然语言处理。令人惊讶的是，高效用并没有提高推理攻击的成功率。
英文摘要：
Texts convey sophisticated knowledge. However, texts also convey sensitive information. Despite the success of general-purpose language models and domain-specific mechanisms with differential privacy (DP), existing text sanitization mechanisms still provide low utility, as cursed by the high-dimensional text representation. The companion issue of utilizing sanitized texts for downstream analytics is also under-explored. This paper takes a direct approach to text sanitization. Our insight is to consider both sensitivity and similarity via our new local DP notion. The sanitized texts also contribute to our sanitization-aware pretraining and fine-tuning, enabling privacy-preserving natural language processing over the BERT language model with promising utility. Surprisingly, the high utility does not boost up the success rate of inference attacks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
