形式与实质:评估大语言模型水印技术的对抗鲁棒性与语言质量平衡
Signature vs. Substance: Evaluating the Balance of Adversarial Resistance and Linguistic Quality in Watermarking Large Language Models
AI总结:
本文针对大语言模型水印技术存在的质量受损、易受对抗攻击的问题,对比评估多种水印技术在两类对抗攻击下的鲁棒性与文本质量保留能力,发现其可保留语义但易受攻击、会偏离原写作风格。
摘要:
为缓解大语言模型(LLMs)生成文本的潜在危害,研究者提出了水印技术,即在文本中嵌入可检测信号的流程,借助水印可始终精准检测LLM生成的文本。但近期研究发现,这类技术往往会损害生成文本的质量,且对抗攻击可剥离水印信号,使文本有可能规避检测,这些问题导致LLM开发者对广泛采用水印技术存在抵触。为推动水印技术落地,本文通过对比释义攻击与回译(即英语→其他语言→英语)攻击,评估了多种水印技术的对抗攻击鲁棒性;同时采用语言指标衡量文本质量与写作风格,评估这些技术保留无水印文本质量与写作风格的能力。实验结果表明,现有水印技术可保留语义,但会偏离无水印文本的写作风格,且易受对抗攻击影响,其中回译攻击的威胁尤为突出。
英文摘要:
To mitigate the potential harms of Large Language Models (LLMs)generated text, researchers have proposed watermarking, a process of embedding detectable signals within text. With watermarking, we can always accurately detect LLM-generated texts. However, recent findings suggest that these techniques often negatively affect the quality of the generated texts, and adversarial attacks can strip the watermarking signals, causing the texts to possibly evade detection. These findings have created resistance in the wide adoption of watermarking by LLM creators. Finally, to encourage adoption, we evaluate the robustness of several watermarking techniques to adversarial attacks by comparing paraphrasing and back translation (i.e., English $\to$ another language $\to$ English) attacks; and their ability to preserve quality and writing style of the unwatermarked texts by using linguistic metrics to capture quality and writing style of texts. Our results suggest that these watermarking techniques preserve semantics, deviate from the writing style of the unwatermarked texts, and are susceptible to adversarial attacks, especially for the back translation attack.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。