arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.18664cs.CLcs.LG
收藏

评估语言模型的版权移除方法

Evaluating Copyright Takedown Methods for Language Models

Princeton University(普林斯顿大学);University of Washington(华盛顿大学)

Boyi Wei, Weijia Shi, Yangsibo Huang, Noah A. Smith, Chiyuan Zhang, Luke Zettlemoyer, Kai Li, Peter Henderson

AI总结:

本文首次评估语言模型版权移除方法的可行性与副作用,提出CoTaEval框架评估移除有效性、事实知识保留及通用效用,发现现有方法均无法在所有指标上表现优异。

摘要:

语言模型(LMs)通过在多样化数据上进行广泛训练来获取其能力,这些数据可能包含受版权保护的材料。这些模型能够记忆并生成与其训练数据相似的内容,从而引发潜在担忧。因此,模型创建者有动力开发缓解方法以防止生成受保护的内容。我们将此过程称为LMs的版权移除,指出其在概念上与DMCA移除相似(但在法律上存在区别)。本文首次对LMs版权移除的可行性和副作用进行了评估。我们提出了CoTaEval,一个评估框架,用于评估版权移除方法的有效性、对模型保留受禁背诵的训练数据中不可版权的事实知识能力的影响,以及模型如何维持其通用效用和效率。我们考察了多种策略,包括添加系统提示、解码时过滤干预和遗忘方法。研究结果表明,没有一种测试方法能在所有指标上表现优异,这表明在这一独特问题场景下仍有巨大的研究空间,并揭示了现行政策提案可能存在未解决的挑战。

英文摘要:

Language models (LMs) derive their capabilities from extensive training on diverse data, including potentially copyrighted material. These models can memorize and generate content similar to their training data, posing potential concerns. Therefore, model creators are motivated to develop mitigation methods that prevent generating protected content. We term this procedure as copyright takedowns for LMs, noting the conceptual similarity to (but legal distinction from) the DMCA takedown This paper introduces the first evaluation of the feasibility and side effects of copyright takedowns for LMs. We propose CoTaEval, an evaluation framework to assess the effectiveness of copyright takedown methods, the impact on the model's ability to retain uncopyrightable factual knowledge from the training data whose recitation is embargoed, and how well the model maintains its general utility and efficiency. We examine several strategies, including adding system prompts, decoding-time filtering interventions, and unlearning approaches. Our findings indicate that no tested method excels across all metrics, showing significant room for research in this unique problem setting and indicating potential unresolved challenges for live policy proposals.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑