使用记忆码的一次性机器遗忘
One-Shot Machine Unlearning with Mnemonic Code
AI总结：
针对现有机器遗忘方法计算成本高、难以应用于ImageNet和Transformer等实际场景的问题，提出一种基于Fisher信息矩阵和记忆码的轻量级方法，通过向敏感参数添加扰动实现快速有效的遗忘。
摘要：
随着深度学习的快速普及，人工智能（AI）应用中固有的伦理和隐私问题日益引起关注。机器遗忘（MU）是通过让已训练的AI模型遗忘不良训练数据来解决这些问题的研究领域。遗憾的是，大多数现有的MU方法在遗忘时会产生大量的时间和计算成本。因此，将这些方法应用于实际数据集和复杂架构（例如ImageNet和Transformer）通常是困难的。为了解决这个问题，我们提出了一种轻量且有效的MU方法。我们的方法识别对遗忘目标敏感的模型参数，并对这些模型参数添加扰动。我们通过计算Fisher信息矩阵（FIM）来识别敏感参数。这种方法不需要为遗忘进行耗时的额外训练。此外，我们引入了称为记忆码的类特定随机信号，以降低FIM计算的成本，因为FIM计算通常需要整个训练数据并产生大量的计算成本。在我们的方法中，我们用记忆码训练模型；在遗忘时，我们使用少量记忆码来计算FIM，并获得用于遗忘的有效扰动。综合实验表明，我们的方法在遗忘方面比现有的MU方法更快、更好。此外，我们表明我们的方法可以扩展到更实际的数据集和复杂的架构。
英文摘要：
Ethical and privacy issues inherent in artificial intelligence (AI) applications have been a growing concern with the rapid spread of deep learning. Machine unlearning (MU) is the research area that addresses these issues by making a trained AI model forget about undesirable training data. Unfortunately, most existing MU methods incur significant time and computational costs for forgetting. Therefore, it is often difficult to apply these methods to practical datasets and sophisticated architectures, e.g., ImageNet and Transformer. To tackle this problem, we propose a lightweight and effective MU method. Our method identifies the model parameters sensitive to the forgetting targets and adds perturbation to such model parameters. We identify the sensitive parameters by calculating the Fisher Information Matrix (FIM). This approach does not require time-consuming additional training for forgetting. In addition, we introduce class-specific random signals called mnemonic code to reduce the cost of FIM calculation, which generally requires the entire training data and incurs significant computational costs. In our method, we train the model with mnemonic code; when forgetting, we use a small number of mnemonic codes to calculate the FIM and get the effective perturbation for forgetting. Comprehensive experiments demonstrate that our method is faster and better at forgetting than existing MU methods. Furthermore, we show that our method can scale to more practical datasets and sophisticated architectures.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
