ArcMark: 通过最优传输实现无失真的多字节大语言模型水印
ArcMark: Distortion-Free Multi-Byte LLM Watermark via Optimal Transport
AI总结：
本文提出ArcMark，一种基于编码和信息论原理的无失真水印方法，能够在数百个token中可靠嵌入多字节信息，且不改变LLM的下一token分布。
摘要：
水印是促进大语言模型（LLM）负责任使用的重要工具。现有水印在生成的token中插入信号，要么标记LLM生成的文本（零比特水印），要么编码更复杂的消息（多比特水印）。尽管最近许多方法在不扰动平均下一token预测的情况下向文本中插入多个比特，但它们很大程度上扩展了零比特设置的设计原则，例如每个token编码单个比特。相比之下，能够将多个字节嵌入文本的水印将极大地增加潜在应用，例如嵌入提交提示的用户ID、使用的精确模型版本，甚至提示本身。我们通过引入ArcMark来解决这个问题：一种基于编码和信息论原理的新型水印构造，能够可靠地将多字节信息嵌入仅几百个token中，而不会对底层LLM的下一token分布造成任何失真。我们通过将无失真水印问题建模为信道编码问题，并推导出信息论信道容量，该容量建立了在LLM输出中无失真嵌入信息的基本极限，从而推导出ArcMark。该容量公式指导了ArcMark的设计。在实践中，ArcMark在重建精度上优于竞争的多比特无失真水印，包括在面对改变部分LLM文本的攻击时。ArcMark输出在困惑度和下游任务质量方面也显示出与未加水印文本无法区分。
英文摘要：
Watermarking is an important tool for promoting the responsible use of large language models (LLMs). Existing watermarks insert a signal into generated tokens that either flags LLM-generated text (zero-bit watermarking) or encodes more complex messages (multi-bit watermarking). Though a number of recent approaches insert multiple bits into text without perturbing average next-token predictions, they largely extend design principles from the zero-bit setting, such as encoding a single bit per token. In contrast, a watermarker capable of embedding multiple bytes into the text would dramatically increase the potential applications, by embedding information such as the ID of the user who submitted the prompt, the precise model version that was used, or even the prompt itself. We address this problem by introducing ArcMark: a new watermark construction based on coding and information-theoretic principles that is capable of reliably embedding multiple bytes of information into just a few hundred tokens, without any distortion of the underlying LLM next-token distribution. We derive ArcMark by formulating the distortion-free watermarking problem as a channel coding problem, and deriving an information-theoretic channel capacity that establishes the fundamental limit of embedding information in LLM output in a distortion-free manner. This capacity formulation informs the design of ArcMark. In practice, ArcMark outperforms competing multi-bit distortion-free watermarks in terms of reconstruction accuracy, including in the face of attacks that alter a subset of the LLM text. ArcMark output is also shown to be indistinguishable from unwatermarked text in terms of perplexity, and in downstream task quality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
