不要遗忘太多:迈向特征层面的机器遗忘
Don't Forget Too Much: Towards Machine Unlearning on Feature Level
AI总结:
针对实例级或类别级遗忘损害效用且粒度过粗的问题,提出特征级机器遗忘框架,分别用对抗学习和模型可解释性筛选加微调处理有无特征标注的场景。
摘要:
机器遗忘使预训练模型能够消除训练数据中某些部分的影响。以往的机器遗忘方案主要集中于遗忘一组实例,或遗忘属于特定类别的全部实例。这些类型的遗忘可能对模型效用产生显著影响;而且,在我们只需要遗忘实例内部的特征、而非整个实例的情况下,它们可能并不适用。由于粒度不同,当前的遗忘方法几乎无法实现特征级遗忘。为了解决效用和粒度方面的挑战,我们提出了一种更精细粒度的遗忘方案,称为“特征遗忘”。我们首先根据是否给出了关于特征的标注信息,探讨了两种不同场景:带已知标注的特征遗忘和不带标注的特征遗忘。关于带已知标注的遗忘,我们提出了一种对抗学习方法,以自动消除与特征有关的影响。对于不带标注的遗忘,我们首先利用模型可解释性技术,使模型某一层的输出能够识别不同的模式特征。随后,我们基于这些具有识别能力的输出,从实例中筛选特征。这样,我们便可以基于筛选后的实例和微调过程消除特征影响。通过在不同场景下、在各种数据集上涉及多种模型的实验,证明了我们所提方法的有效性。
英文摘要:
Machine unlearning enables pre-trained models to remove the effect of certain portions of training data. Previous machine unlearning schemes have mainly focused on unlearning a cluster of instances or all instances belonging to a specific class. These types of unlearning might have a significant impact on the model utility; and they may be inadequate for situations where we only need to unlearn features within instances, rather than the whole instances. Due to the different granularity, current unlearning methods can hardly achieve feature-level unlearning. To address the challenges of utility and granularity, we propose a refined granularity unlearning scheme referred to as ``feature unlearning". We first explore two distinct scenarios based on whether the annotation information about the features is given: feature unlearning with known annotations and feature unlearning without annotations. Regarding unlearning with known annotations, we propose an adversarial learning approach to automatically remove effects about features. For unlearning without annotations, we initially enable the output of one model's layer to identify different pattern features using model interpretability techniques. We proceed to filter features from instances based on these outputs with identifying ability. So that we can remove the feature impact based on filtered instances and the fine-tuning process. The effectiveness of our proposed approach is demonstrated through experiments involving diverse models on various datasets in different scenarios.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。