我们真的在遗忘吗？对机器遗忘评估协议的批判性重新审视
Are We Truly Forgetting? A Critical Re-examination of Machine Unlearning Evaluation Protocols
AI总结：
本文批判性地重新审视了机器遗忘评估协议，提出了一种大规模场景下的表示基于评估方法，揭示现有方法在表示质量上的不足，并引入新的评估场景以更全面地评估遗忘算法。
摘要：
机器遗忘是一种在保持保留数据性能的同时从训练模型中移除特定数据点的过程，以应对隐私或法律要求。尽管其重要性，现有的遗忘评估往往集中在小规模场景下的logit基于指标。我们发现这可能导致在现实场景中遗忘方法产生虚假的安全感。在本文中，我们进行了全面的评估，采用大规模场景下的表示基于评估来验证遗忘方法是否真正从模型的表示角度消除了目标数据。我们的分析揭示了当前最先进的遗忘方法要么完全降级了遗忘模型的表示质量，要么仅仅修改分类器，从而在保持与原始模型相似的表示的同时实现优越的logit基于性能。此外，我们引入了一个新的遗忘评估场景，其中遗忘类别与下游任务类别具有语义相似性，这要求特征表示显著偏离原始模型的表示，从而从表示角度进行更彻底的评估。我们希望我们的基准将为在现实条件下评估遗忘算法提供标准化的协议。
英文摘要：
Machine unlearning is a process to remove specific data points from a trained model while maintaining the performance on the retain data, addressing privacy or legal requirements. Despite its importance, existing unlearning evaluations tend to focus on logit-based metrics under small-scale scenarios. We observe that this could lead to a false sense of security in unlearning approaches under real-world scenarios. In this paper, we conduct a comprehensive evaluation that employs representation-based evaluations of the unlearned model under large-scale scenarios to verify whether the unlearning approaches truly eliminate the targeted data from the model's representation perspective. Our analysis reveals that current state-of-the-art unlearning approaches either completely degrade the representational quality of the unlearned model or merely modify the classifier, thereby achieving superior logit-based performance while maintaining representational similarity to the original model. Furthermore, we introduce a novel unlearning evaluation scenario in which the forgetting classes exhibit semantic similarity to downstream task classes, necessitating that feature representations diverge significantly from those of the original model, thus enabling a more thorough evaluation from a representation perspective. We hope our benchmark will serve as a standardized protocol for evaluating unlearning algorithms under realistic conditions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
