大语言模型中的数字遗忘：遗忘方法综述
Digital Forgetting in Large Language Models: A Survey of Unlearning Methods
AI总结：
本综述聚焦大语言模型的数字遗忘问题，梳理其背景、动机与特性，详细分类并对比现有机器遗忘方法，介绍评估所用的数据集、模型与指标，同时讨论领域挑战并给出总结。
摘要：
数字遗忘的目标是，给定一个存在不良知识或行为的模型，得到一个已消除所检测问题的新模型。遗忘的动机包括隐私保护、版权保护、消除偏见与歧视，以及防止有害内容生成。有效的数字遗忘必须具备有效性（即新模型对不良知识/行为的遗忘程度）、保留原模型在期望任务上的性能，以及可扩展性（尤其是遗忘的效率必须高于仅在需保留的任务/数据上从头重新训练）。本综述聚焦于大语言模型（LLMs）中的遗忘问题。我们首先介绍LLMs的背景，包括其组成部分、LLMs的类型以及其常规训练流程。其次，我们阐述数字遗忘的动机、类型和期望特性。第三，我们介绍LLMs中的数字遗忘方法，其中遗忘（unlearning）方法是当前最先进的技术。第四，我们对LLMs的机器遗忘方法进行了详细分类，并调研和比较了现有方法。第五，我们详述了用于评估遗忘效果、保留性能和运行时的数据集、模型和指标。第六，我们讨论了该领域面临的挑战。最后，我们给出了一些总结性意见。
英文摘要：
The objective of digital forgetting is, given a model with undesirable knowledge or behavior, obtain a new model where the detected issues are no longer present. The motivations for forgetting include privacy protection, copyright protection, elimination of biases and discrimination, and prevention of harmful content generation. Effective digital forgetting has to be effective (meaning how well the new model has forgotten the undesired knowledge/behavior), retain the performance of the original model on the desirable tasks, and be scalable (in particular forgetting has to be more efficient than retraining from scratch on just the tasks/data to be retained). This survey focuses on forgetting in large language models (LLMs). We first provide background on LLMs, including their components, the types of LLMs, and their usual training pipeline. Second, we describe the motivations, types, and desired properties of digital forgetting. Third, we introduce the approaches to digital forgetting in LLMs, among which unlearning methodologies stand out as the state of the art. Fourth, we provide a detailed taxonomy of machine unlearning methods for LLMs, and we survey and compare current approaches. Fifth, we detail datasets, models and metrics used for the evaluation of forgetting, retaining and runtime. Sixth, we discuss challenges in the area. Finally, we provide some concluding remarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
