遗忘有邻居：机器遗忘中的局部连带遗忘
Forgetting Has Neighbors: Localized Collateral Forgetting in Machine Unlearning
AI总结：
本文研究机器遗忘中梯度上升和随机标签方法导致的局部连带遗忘现象，并提出了基于局部教师蒸馏的缓解策略。
摘要：
机器遗忘旨在无需完全重新训练的情况下移除选定训练样本的影响。标准评估通常使用聚合指标（如准确率和遗忘分数）来概括遗忘质量，这可能会掩盖局部失败。我们通过比较遗忘模型与删除后重新训练模型的预测，在样本级别研究这种失败模式。我们表明，这种逐点差异可能高度不均匀：对于梯度上升和随机标签方法，无论是否进行保留集微调，差异都随着与遗忘集的几何接近度而增大。我们将这种现象称为局部连带遗忘。我们的分析确定了该效应背后的机制：遗忘过程中使用的替代目标可能与重新训练引起的局部预测结构不一致，并且这种不一致通过共享表示传播到邻近样本。受此机制启发，我们提出了局部教师蒸馏，一种简单的缓解策略，用仅在遗忘集的保留邻居上训练的小教师生成的软标签替换随机目标。在CIFAR-100部分类别删除任务中，这种局部教师使遗忘模型更接近重新训练，尤其是在遗忘集附近，同时保持有竞争力的聚合遗忘指标。
英文摘要：
Machine unlearning aims to remove the influence of selected training examples without full retraining. Standard evaluations often summarize unlearning quality with aggregate metrics, such as accuracy- and forgetting-based scores, which can hide localized failures. We study this failure mode at the example level by comparing the predictions of an unlearned model to those of the model retrained after deletion. We show that this pointwise discrepancy can be highly non-uniform: for gradient-ascent and random-labeling methods, with and without retain-set fine-tuning, it grows with geometric proximity to the forget set. We call this phenomenon localized collateral forgetting. Our analysis identifies a mechanism behind the effect: surrogate targets used during unlearning can be inconsistent with the local prediction structure induced by retraining, and this inconsistency propagates through shared representations to nearby examples. Motivated by this mechanism, we propose Local Teacher Distillation, a simple mitigation strategy that replaces random targets with soft labels from a small teacher trained only on retained neighbors of the forget set. On CIFAR-100 partial-class deletion, this local teacher brings the unlearned model substantially closer to retraining, especially near the forget set, while maintaining competitive aggregate unlearning metrics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
