arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.10952cs.CL
收藏

通过大语言模型遗忘避免版权侵权

Avoiding Copyright Infringement via Large Language Model Unlearning

University of Pennsylvania(宾夕法尼亚大学);University of Notre Dame(圣母大学);Northwestern University(西北大学)

Guangyao Dou, Zheyuan Liu, Qing Lyu, Kaize Ding, Eric Wong

AI总结:

针对LLMs因生成受版权保护内容引发的问题,提出稳定序列遗忘(SSU)框架,通过识别移除对应权重更新、引入随机标签损失等方法,在遗忘效果与通用能力间实现有效权衡,优于现有基线。

摘要:

预训练大语言模型(LLMs)展现出卓越能力,但也因学习和生成受版权保护的材料而带来风险,引发重大法律和伦理担忧。在现实场景中,随着不同时间点出现新的内容移除请求,模型所有者需持续应对版权侵权问题。这导致对序列遗忘的需求,即随着新请求出现,受版权保护的内容被依次移除。尽管具有实际相关性,但现有文献中尚未严格探索版权侵权语境下的序列遗忘。为填补这一空白,我们提出稳定序列遗忘(SSU),这是一个旨在通过多个时间步从LLMs中遗忘受版权保护内容的新型框架。我们的方法通过识别并移除模型参数中对应受版权保护内容的特定权重更新来工作。我们通过引入随机标签损失提高遗忘效果,并通过调整目标参数确保模型保留通用知识。实验结果表明,SSU在遗忘效果和通用语言能力之间实现了有效的权衡,优于现有基线。

英文摘要:

Pre-trained Large Language Models (LLMs) have demonstrated remarkable capabilities but also pose risks by learning and generating copyrighted material, leading to significant legal and ethical concerns. In real-world scenarios, model owners need to continuously address copyright infringement as new requests for content removal emerge at different time points. This leads to the need for sequential unlearning, where copyrighted content is removed sequentially as new requests arise. Despite its practical relevance, sequential unlearning in the context of copyright infringement has not been rigorously explored in existing literature. To address this gap, we propose Stable Sequential Unlearning (SSU), a novel framework designed to unlearn copyrighted content from LLMs over multiple time steps. Our approach works by identifying and removing specific weight updates in the model's parameters that correspond to copyrighted content. We improve unlearning efficacy by introducing random labeling loss and ensuring the model retains its general-purpose knowledge by adjusting targeted parameters. Experimental results show that SSU achieves an effective trade-off between unlearning efficacy and general-purpose language abilities, outperforming existing baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑