WatME:通过词汇冗余实现无损水印
WatME: Towards Lossless Watermarking Through Lexical Redundancy
AI总结:
本文提出WatME方法,利用LLM词汇表中的固有词汇冗余和互斥规则动态优化解码词元选择,在保持水印可检测性的同时减少对模型知识回忆、逻辑推理和语言生成能力的损害,实现接近无损的文本水印嵌入。
摘要:
文本水印已成为识别机器生成文本的关键技术。然而,现有方法在解码过程中往往依赖任意的词汇表划分来嵌入水印,这损害了合适词元的可用性,并显著降低了回复的质量。本研究从认知科学的视角评估了水印对大型语言模型(LLMs)不同能力的影响。我们的发现揭示了一个显著差异:知识回忆和逻辑推理比语言生成受到的不利影响更大。这些结果表明,水印对LLMs的影响比此前理解的更为深远。为应对这些挑战,我们提出了互斥水印(WatME),这是一种利用LLM词汇表中固有词汇冗余的语言先验知识来无缝集成水印的新方法。具体而言,WatME通过对识别出的词汇冗余应用互斥规则,在解码过程中动态优化词元使用。该策略有效防止了合适词元不可用的情况,并保持了LLMs的表达能力。我们提供了理论分析和实证证据,表明WatME在确保水印可检测性的同时,有效保持了LLMs的多样化能力。
英文摘要:
Text watermarking has emerged as a pivotal technique for identifying machine-generated text. However, existing methods often rely on arbitrary vocabulary partitioning during decoding to embed watermarks, which compromises the availability of suitable tokens and significantly degrades the quality of responses. This study assesses the impact of watermarking on different capabilities of large language models (LLMs) from a cognitive science lens. Our finding highlights a significant disparity; knowledge recall and logical reasoning are more adversely affected than language generation. These results suggest a more profound effect of watermarking on LLMs than previously understood. To address these challenges, we introduce Watermarking with Mutual Exclusion (WatME), a novel approach leveraging linguistic prior knowledge of inherent lexical redundancy in LLM vocabularies to seamlessly integrate watermarks. Specifically, WatME dynamically optimizes token usage during the decoding process by applying a mutually exclusive rule to the identified lexical redundancies. This strategy effectively prevents the unavailability of appropriate tokens and preserves the expressive power of LLMs. We provide both theoretical analysis and empirical evidence showing that WatME effectively preserves the diverse capabilities of LLMs while ensuring watermark detectability.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。