通过选择性突触抑制实现无需重训练的快速机器遗忘
Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampening
AI总结：
针对现有机器遗忘方法需重训练或计算成本高的问题，提出无需重训练的两步式事后方法SSD，通过Fisher信息矩阵选参并按重要性抑制，性能媲美重训练方法。
摘要：
机器遗忘，即机器学习模型的遗忘能力，在遵守数据隐私法规以及移除有害、被操纵或过时信息方面正变得愈发重要。其核心挑战在于，既要遗忘特定信息，又要保障模型在剩余数据上的性能。尽管当前的前沿方法表现出色，但它们通常需要对保留数据进行一定程度的重训练，以维护或恢复模型性能。这会增加计算开销，并且要求训练数据始终可用且可访问，而这一点可能无法实现。相比之下，其他方法采用无需重训练的范式，但这类方法的计算成本高得令人难以承受，且性能不及基于重训练的同类方法。我们提出了选择性突触抑制（Selective Synaptic Dampening，SSD），这是一种新颖的两步式事后无需重训练的机器遗忘方法，速度快、性能佳，且无需长期存储训练数据。首先，SSD利用训练数据和遗忘数据的Fisher信息矩阵，筛选出对遗忘集格外重要的参数。其次，SSD根据这些参数相对于更广泛训练数据而言对遗忘集的相对重要性，按比例抑制这些参数，从而实现遗忘。我们使用ResNet18和Vision Transformer开展了一系列实验，将我们的方法与几种现有的遗忘方法进行对比。结果表明，SSD的性能与基于重训练的事后方法具有竞争力，证明了无需重训练的事后遗忘方法的可行性。
英文摘要：
Machine unlearning, the ability for a machine learning model to forget, is becoming increasingly important to comply with data privacy regulations, as well as to remove harmful, manipulated, or outdated information. The key challenge lies in forgetting specific information while protecting model performance on the remaining data. While current state-of-the-art methods perform well, they typically require some level of retraining over the retained data, in order to protect or restore model performance. This adds computational overhead and mandates that the training data remain available and accessible, which may not be feasible. In contrast, other methods employ a retrain-free paradigm, however, these approaches are prohibitively computationally expensive and do not perform on par with their retrain-based counterparts. We present Selective Synaptic Dampening (SSD), a novel two-step, post hoc, retrain-free approach to machine unlearning which is fast, performant, and does not require long-term storage of the training data. First, SSD uses the Fisher information matrix of the training and forgetting data to select parameters that are disproportionately important to the forget set. Second, SSD induces forgetting by dampening these parameters proportional to their relative importance to the forget set with respect to the wider training data. We evaluate our method against several existing unlearning methods in a range of experiments using ResNet18 and Vision Transformer. Results show that the performance of SSD is competitive with retrain-based post hoc methods, demonstrating the viability of retrain-free post hoc unlearning approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
