迈向自然机器遗忘
Towards Natural Machine Unlearning
AI总结：
本文提出一种自然机器遗忘方法，通过向遗忘样本注入正确信息并配对标签，显著减少过度遗忘并增强超参数鲁棒性，优于现有最先进方法。
摘要：
机器遗忘（MU）旨在从预训练模型中消除从特定训练数据（即遗忘数据）中学到的信息。目前，现有MU方法的主流做法涉及用错误标签修改遗忘数据，随后对模型进行微调。虽然学习此类错误信息确实可以移除知识，但该过程相当不自然，因为遗忘过程会不必要地强化错误信息并导致过度遗忘。为了迈向更“自然”的机器遗忘，我们在更改遗忘样本标签时，将来自剩余数据的正确信息注入到遗忘样本中。通过将这些调整后的样本与其标签配对，模型将倾向于使用注入的正确信息，并自然地抑制本应被遗忘的信息。尽管方法简单，但这一迈向自然机器遗忘的第一步可以显著优于当前最先进的方法。特别是，我们的方法大幅减少了过度遗忘，并对超参数表现出强大的鲁棒性，使其成为实际机器遗忘的有力候选方案。
英文摘要：
Machine unlearning (MU) aims to eliminate information that has been learned from specific training data, namely forgetting data, from a pre-trained model. Currently, the mainstream of existing MU methods involves modifying the forgetting data with incorrect labels and subsequently fine-tuning the model. While learning such incorrect information can indeed remove knowledge, the process is quite unnatural as the unlearning process undesirably reinforces the incorrect information and leads to over-forgetting. Towards more \textit{natural} machine unlearning, we inject correct information from the remaining data to the forgetting samples when changing their labels. Through pairing these adjusted samples with their labels, the model will tend to use the injected correct information and naturally suppress the information meant to be forgotten. Albeit straightforward, such a first step towards natural machine unlearning can significantly outperform current state-of-the-art approaches. In particular, our method substantially reduces the over-forgetting and leads to strong robustness to hyperparameters, making it a promising candidate for practical machine unlearning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
