锚定解码:可证明降低任何语言模型的版权风险
Anchored Decoding: Provably Reducing Copyright Risk for Any Language Model
AI总结:
提出锚定解码,一种即插即用的推理时方法,通过将生成内容约束在许可训练的安全模型附近,可证明地抑制语言模型逐字复制受版权保护的内容,实现可调的风险-效用权衡。
摘要:
语言模型倾向于记忆其训练数据的部分内容并逐字生成。当底层来源敏感或受版权保护时,这种复现会引发创作者同意和补偿问题以及开发者合规风险。我们提出锚定解码,一种即插即用的推理时方法,用于抑制逐字复制:它通过将生成内容保持在许可训练的安全模型的有界邻近范围内,使得任何在混合许可数据上训练的有风险语言模型都能进行解码。锚定解码在生成轨迹上自适应地分配用户选择的信息预算,并强制执行每步约束,从而提供序列级别的保证,实现可调的风险-效用权衡。为使锚定解码实用化,我们引入了一个新的许可训练的安全模型(TinyComma 1.8B),以及锚定字节解码,这是我们方法的字节级变体,通过ByteSampler框架(Hayase等人,2025)实现跨词汇融合。在六个模型对上,针对复制风险和效用的长文本指标,锚定解码和锚定字节解码定义了新的帕累托前沿,在保持接近原始流畅性和事实性的同时,将有风险基线与安全参考之间的可测量复制差距缩小了高达75%,且推理开销适中。
英文摘要:
Language models (LMs) tend to memorize portions of their training data and emit verbatim spans. When the underlying sources are sensitive or copyright-protected, such reproduction raises issues of consent and compensation for creators and compliance risks for developers. We propose Anchored Decoding, a plug-and-play inference-time method for suppressing verbatim copying: it enables decoding from any risky LM trained on mixed-license data by keeping generation in bounded proximity to a permissively trained safe LM. Anchored Decoding adaptively allocates a user-chosen information budget over the generation trajectory and enforces per-step constraints that yield a sequence-level guarantee, enabling a tunable risk-utility trade-off. To make Anchored Decoding practically useful, we introduce a new permissively trained safe model (TinyComma 1.8B), as well as Anchored$_{\mathrm{Byte}}$ Decoding, a byte-level variant of our method that enables cross-vocabulary fusion via the ByteSampler framework (Hayase et al., 2025). Across six model pairs on long-form metrics for copying risk and utility, Anchored and Anchored$_{\mathrm{Byte}}$ Decoding define a new Pareto frontier, preserving near-original fluency and factuality while closing up to 75% of the measurable copying gap between the risky baseline and a safe reference, at a modest inference overhead.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。