擦除，但未消失：输出遗忘并非真正遗忘
Erased, but Not Gone: Output Forgetting Is Not True Forgetting
AI总结：
本文通过重训练一致性表示遗忘框架，揭示当前机器遗忘评估中输出级遗忘可能高估遗忘效果，实际存在结构化残留。
摘要：
机器遗忘通常通过输出遗忘来评判，例如低遗忘集准确率或降低的logit级成员推断。但是，如果输出级成功可以与表示空间中与重训练不一致的残留共存，那么当前评估实际上在认证什么样的遗忘？我们通过重训练一致的表示遗忘来研究这个问题，使用重训练模型（即从头训练，不包含遗忘数据）作为正确遗忘的操作参考。跨多种遗忘方法、数据集和模型，我们的理论分析和实证结果表明，标准输出级评估可能系统性地高估遗忘的成功。在这种更强的视角下，当前方法通常在输出层看起来被遗忘了，但相对于重训练表现出结构化的不匹配。它们在遗忘样本上与重训练部分对齐，在保留样本上更不一致，并且残留差异集中在与重训练相关的方向上，而不是在表示空间中扩散。这种结构化的不匹配通过遗忘/保留不对称、方向性不匹配以及沿重训练相关方向的集中残留来表征。这些结果表明，当前的机器遗忘通常被评估为表面遗忘，而非重训练一致的遗忘。更广泛地说，重训练揭示了输出遗忘所隐藏的东西。
英文摘要：
Machine unlearning (MU) is commonly judged by output forgetting, such as low forget-set accuracy or reduced membership inference. But if output-level success can coexist with retraining-inconsistent residuals in representation space, what kind of forgetting are current evaluations actually certifying? We study this question through retraining-consistent representation forgetting, using a population of retrained models (i.e., trained from scratch without the forget data) as an operational reference distribution for ideal forgetting. Across multiple unlearning methods, datasets, and models, our theoretical analysis and empirical results show that standard output-level evaluation can systematically overestimate the success of unlearning. Under this stronger lens, current methods often appear forgotten at the output layer while exhibiting a structured mismatch relative to retraining. They partially align with retraining on forget samples, remain more inconsistent on retain samples, and tend to leave residual discrepancy localized along retraining-related directions rather than diffuse in representation space. This structured mismatch is characterized by forget/retain asymmetry, directional mismatch, and concentrated residuals along retraining-related directions. In many cases, the discrepancies exceed the variability of independent retraining and lead to a risk of information recovery during relearning attacks. Overall, the results suggest that current MU is often evaluated for apparent forgetting rather than retraining-consistent forgetting. More broadly, retraining reveals what output forgetting hides.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
