GumbelSoft:通过GumbelMax技巧实现多样化的语言模型水印
GumbelSoft: Diversified Language Model Watermarking via the GumbelMax-trick
AI总结:
针对GM水印生成多样性不足的问题,提出Logits-Addition水印及其变体,其中GumbelSoft变体在高多样性下AUROC分数优于其他方法0.1至0.3。
摘要:
大型语言模型(LLMs)擅长生成类似人类的文本,但也引发了对虚假新闻和学术不端行为滥用的担忧。基于解码的水印,特别是基于GumbelMax技巧的水印(GM水印),因其显著的检测能力而成为保护机器生成文本的突出解决方案。然而,GM水印在生成多样性方面面临重大挑战,对于相同的提示总是产生相同的输出,对生成多样性和用户体验产生负面影响。为了克服这一限制,我们提出了一种新型的GM水印,即Logits-Addition水印,及其三个变体,专门设计用于增强多样性。其中,GumbelSoft水印(Logits-Addition水印的softmax变体)在高多样性设置中表现出优越的性能,其AUROC分数比另外两个替代变体高出0.1到0.3,并且比其他基于解码的水印方法至少高出0.1。
英文摘要:
Large language models (LLMs) excellently generate human-like text, but also raise concerns about misuse in fake news and academic dishonesty. Decoding-based watermark, particularly the GumbelMax-trick-based watermark(GM watermark), is a standout solution for safeguarding machine-generated texts due to its notable detectability. However, GM watermark encounters a major challenge with generation diversity, always yielding identical outputs for the same prompt, negatively impacting generation diversity and user experience. To overcome this limitation, we propose a new type of GM watermark, the Logits-Addition watermark, and its three variants, specifically designed to enhance diversity. Among these, the GumbelSoft watermark (a softmax variant of the Logits-Addition watermark) demonstrates superior performance in high diversity settings, with its AUROC score outperforming those of the two alternative variants by 0.1 to 0.3 and surpassing other decoding-based watermarking methods by a minimum of 0.1.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。