arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.16634cs.CYcs.AIcs.CR
收藏

RLCP:一种基于强化学习的文本到图像扩散模型版权保护方法

RLCP: A Reinforcement Learning-based Copyright Protection Method for Text-to-Image Diffusion Model

Zhuan Shi, Jing Yan, Xiaoli Tang, Lingjuan Lyu, Boi Faltings

AI总结:

针对文本到图像扩散模型的版权保护挑战,提出RLCP方法:基于版权法和判例构建版权指标,结合DDPO框架与KL散度正则化,在保持图像质量的同时显著降低侵权风险。

摘要:

文本到图像生成模型的日益复杂,导致在定义和执行版权侵权标准与保护方面出现了复杂挑战。现有方法(如水印和数据集去重)由于缺乏标准化指标以及应对扩散模型中版权侵权的固有复杂性,无法提供全面解决方案。为应对这些挑战,我们提出了一种基于强化学习的文本到图像扩散模型版权保护(RLCP)方法,该方法在保持模型生成数据集质量的同时,最大限度地减少侵权内容的生成。我们的方法首先引入一种基于版权法和侵权法庭判例的新型版权指标;然后利用去噪扩散策略优化(DDPO)框架引导模型进行多步决策,使用包含所提版权指标的奖励函数对其优化;此外,采用KL散度作为正则化项以缓解部分失效模式并稳定强化学习微调。在3个版权与非版权图像混合数据集上的实验表明,该方法在保持图像质量的同时显著降低了版权侵权风险。

英文摘要:

The increasing sophistication of text-to-image generative models has led to complex challenges in defining and enforcing copyright infringement criteria and protection. Existing methods, such as watermarking and dataset deduplication, fail to provide comprehensive solutions due to the lack of standardized metrics and the inherent complexity of addressing copyright infringement in diffusion models. To deal with these challenges, we propose a Reinforcement Learning-based Copyright Protection(RLCP) method for Text-to-Image Diffusion Model, which minimizes the generation of copyright-infringing content while maintaining the quality of the model-generated dataset. Our approach begins with the introduction of a novel copyright metric grounded in copyright law and court precedents on infringement. We then utilize the Denoising Diffusion Policy Optimization (DDPO) framework to guide the model through a multi-step decision-making process, optimizing it using a reward function that incorporates our proposed copyright metric. Additionally, we employ KL divergence as a regularization term to mitigate some failure modes and stabilize RL fine-tuning. Experiments conducted on 3 mixed datasets of copyright and non-copyright images demonstrate that our approach significantly reduces copyright infringement risk while maintaining image quality.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑