LUME:基于多任务评估的大语言模型遗忘
LUME: LLM Unlearning with Multitask Evaluations
AI总结:
本文提出多任务遗忘基准LUME,涵盖三类任务,并发布1B和7B参数的目标模型,系统评估多种遗忘算法,揭示其行为与局限。
摘要:
遗忘旨在无需完全重新训练的情况下,从大型语言模型(LLMs)中移除受版权保护、敏感或私有的内容。在这项工作中,我们开发了一个多任务遗忘基准(LUME),其包含三个任务:(1)遗忘合成生成的创意短篇小说,(2)遗忘带有敏感信息的合成传记,以及(3)遗忘一组公开传记。我们进一步发布了两个微调后的大小分别为1B和7B参数规模的LLM作为目标模型。我们对几种近期提出的遗忘算法进行了详细评估,并在精心设计的指标上呈现结果,以理解它们的行为和局限性。
英文摘要:
Unlearning aims to remove copyrighted, sensitive, or private content from large language models (LLMs) without a full retraining. In this work, we develop a multi-task unlearning benchmark (LUME) which features three tasks: (1) unlearn synthetically generated creative short novels, (2) unlearn synthetic biographies with sensitive information, and (3) unlearn a collection of public biographies. We further release two fine-tuned LLMs of 1B and 7B parameter sizes as the target models. We conduct detailed evaluations of several recently proposed unlearning algorithms and present results on carefully crafted metrics to understand their behavior and limitations.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。