arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2512.06993cs.LG
收藏

迈向可靠的机器遗忘:理论、算法与评估

Toward Reliable Machine Unlearning: Theory, Algorithms, and Evaluation

Computer Science(计算机科学)

Ali Ebrahimpour-Boroojeny

AI总结:

本文提出AMUN和TRW方法,通过对抗性技术提升机器遗忘的可靠性与性能。

摘要:

我们提出了新的方法用于随机样本集的遗忘和类别遗忘,并展示了这些方法优于现有方法。我们的遗忘方法的主要驱动力是重新训练模型在遗忘样本和保留样本上的预测相似性。我们引入了对抗性机器遗忘(AMUN),它在图像分类上基于最先进的MIA分数超越了现有最先进方法。AMUN通过在对应的对抗示例上微调来降低模型对遗忘样本的置信度。通过理论分析,我们识别了影响AMUN性能的因素,包括平滑性。为了促进具有受控Lipschitz常数的平滑模型的训练,我们提出了FastClip,一种可扩展的方法,它对仿射层执行逐层谱范数裁剪。在另一项研究中,我们表明增加平滑性自然提高了对抗示例转移,从而支持上述第二个因素。遵循相同的原理进行类别遗忘,我们证明现有方法在复制重新训练模型的行为时失败,通过引入一个最近邻成员推断攻击(MIA-NN),该攻击利用分配给邻近类别的概率来检测被遗忘的样本,并展示此类方法的脆弱性。我们然后提出了一种微调目标,通过近似对于遗忘类输入,重新训练从头开始的模型所产生的剩余类别的分布来缓解这种泄漏。为了构建这个近似,我们估计类间相似性并相应地调整目标模型的分布。所得到的倾斜再加权(TRW)分布在微调过程中作为所需的目标。在多个基准测试中,TRW在先前指标上匹配或超越现有遗忘方法。

英文摘要:

We propose new methodologies for both unlearning random set of samples and class unlearning and show that they outperform existing methods. The main driver of our unlearning methods is the similarity of predictions to a retrained model on both the forget and remain samples. We introduce Adversarial Machine UNlearning (AMUN), which surpasses prior state-of-the-art methods for image classification based on SOTA MIA scores. AMUN lowers the model's confidence on forget samples by fine-tuning on their corresponding adversarial examples. Through theoretical analysis, we identify factors governing AMUN's performance, including smoothness. To facilitate training of smooth models with a controlled Lipschitz constant, we propose FastClip, a scalable method that performs layer-wise spectral-norm clipping of affine layers. In a separate study, we show that increased smoothness naturally improves adversarial example transfer, thereby supporting the second factor above. Following the same principles for class unlearning, we show that existing methods fail in replicating a retrained model's behavior by introducing a nearest-neighbor membership inference attack (MIA-NN) that uses the probabilities assigned to neighboring classes to detect unlearned samples and demonstrate the vulnerability of such methods. We then propose a fine-tuning objective that mitigates this leakage by approximating, for forget-class inputs, the distribution over remaining classes that a model retrained from scratch would produce. To construct this approximation, we estimate inter-class similarity and tilt the target model's distribution accordingly. The resulting Tilted ReWeighting(TRW) distribution serves as the desired target during fine-tuning. Across multiple benchmarks, TRW matches or surpasses existing unlearning methods on prior metrics.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑