MEOW:基于反向事实的记忆监督大语言模型遗忘
MEOW: MEMOry Supervised LLM Unlearning Via Inverted Facts
AI总结:
提出MEOW,一种基于梯度下降的LLM遗忘方法,利用离线生成的反向事实和记忆度量MEMO选择数据微调,在ToFU基准上提升遗忘质量且保持模型实用性。
摘要:
大语言模型(LLMs)可能记忆敏感信息,引发对其潜在滥用的担忧。LLM遗忘是一种事后方法,旨在从已训练的模型中移除这些信息,为缓解这些风险提供了一种有前景的解决方案。然而,以往的做法面临三个关键挑战:1. 实用性:成功的遗忘常常导致在无关任务上发生灾难性崩溃。2. 效率:许多方法要么涉及添加规模相似的模型,这会减慢遗忘或推理速度,要么需要难以获取的保留数据。3. 鲁棒性:即使有效的方法仍可能通过提取技术泄露数据。为解决这些挑战,我们提出了MEOW,一种简单而有效的基于梯度下降的遗忘方法。具体来说,我们使用一个离线LLM生成一组反向事实。然后,我们设计了一个新指标MEMO,用于量化LLM中的记忆程度。最后,基于MEMO提供的信号,我们选择最合适的反向事实集,并基于它们对模型进行微调。我们在常用的遗忘基准ToFU上,使用Llama2-7B-Chat和Phi-1.5B评估了MEOW,并在NLU和NLG任务上进行了测试。结果表明,MEOW在遗忘质量上显著提升,而模型实用性没有大幅损失。同时,MEOW在NLU或NLG能力上没有表现出显著退化,甚至在NLU性能上略有提升。
英文摘要:
Large Language Models (LLMs) can memorize sensitive information, raising concerns about potential misuse. LLM Unlearning, a post-hoc approach to remove this information from trained LLMs, offers a promising solution to mitigate these risks. However, previous practices face three key challenges: 1. Utility: successful unlearning often causes catastrophic collapse on unrelated tasks. 2. Efficiency: many methods either involve adding similarly sized models, which slows down unlearning or inference, or require retain data that are difficult to obtain. 3. Robustness: even effective methods may still leak data via extraction techniques. To address these challenges, we propose MEOW, a simple yet effective gradient descent-based unlearning method. Specifically, we use an offline LLM to generate a set of inverted facts. Then, we design a new metric, MEMO, to quantify memorization in LLMs. Finally, based on the signals provided by MEMO, we select the most appropriate set of inverted facts and finetune the model based on them. We evaluate MEOW on the commonly used unlearn benchmark, ToFU, with Llama2-7B-Chat and Phi-1.5B, and test it on both NLU and NLG tasks. Results demonstrate significant improvement of MEOW in forget quality without substantial loss in model utility. Meanwhile, MEOW does not exhibit significant degradation in NLU or NLG capabilities, and there is even a slight improvement in NLU performance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。