JAMDEC:基于小语言模型受限解码的无监督作者身份混淆
JAMDEC: Unsupervised Authorship Obfuscation using Constrained Decoding over Small Language Models
AI总结:
本文提出无监督推理时作者身份混淆算法JAMDEC,基于GPT2-XL等小语言模型通过受限解码提升创造力,在保护隐私的同时优于同等小模型方法,并与大两数量级的GPT3.5竞争。
摘要:
在线内容的持久性与不断增强的作者身份识别技术相结合,要求在需要时采用更强大的计算方法来保护在线作者的身份与隐私,例如科学论文的盲审、匿名在线评论,或心理健康论坛中的匿名互动。在本文中,我们提出了一种用于作者身份混淆的无监督推理时方法,以应对作者身份混淆的独特挑战:缺乏针对多样化作者身份和领域的监督数据,以及需要超越简单复述的充分修订水平来混淆作者身份,同时保留原始内容和流畅性。我们引入了JAMDEC,这是一种用户可控的推理时作者身份混淆算法,原则上可应用于任何文本和作者身份。我们的方法基于GPT2-XL等小语言模型构建,以帮助避免向专有LLM的API披露原始内容,同时通过算法增强缩小小语言模型与大语言模型之间的性能差距。我们方法背后的关键思想是通过受限解码提升较小语言模型的创造力,同时允许用户指定的控制和灵活性。实验结果表明,我们基于GPT2-XL的方法优于基于同等较小模型的先前最先进方法,同时与GPT3.5 175B(一个大两个数量级的专有模型)相比具有竞争力。
英文摘要:
The permanence of online content combined with the enhanced authorship identification techniques calls for stronger computational methods to protect the identity and privacy of online authorship when needed, e.g., blind reviews for scientific papers, anonymous online reviews, or anonymous interactions in the mental health forums. In this paper, we propose an unsupervised inference-time approach to authorship obfuscation to address the unique challenges of authorship obfuscation: lack of supervision data for diverse authorship and domains, and the need for a sufficient level of revision beyond simple paraphrasing to obfuscate the authorship, all the while preserving the original content and fluency. We introduce JAMDEC, a user-controlled, inference-time algorithm for authorship obfuscation that can be in principle applied to any text and authorship. Our approach builds on small language models such as GPT2-XL in order to help avoid disclosing the original content to proprietary LLM's APIs, while also reducing the performance gap between small and large language models via algorithmic enhancement. The key idea behind our approach is to boost the creative power of smaller language models through constrained decoding, while also allowing for user-specified controls and flexibility. Experimental results demonstrate that our approach based on GPT2-XL outperforms previous state-of-the-art methods based on comparably small models, while performing competitively against GPT3.5 175B, a propriety model that is two orders of magnitudes larger.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。