标签无关遗忘：深度模型中的无监督遗忘方法
Label-Agnostic Forgetting: A Supervision-Free Unlearning in Deep Models
AI总结：
本文提出 Label-Agnostic Forgetting（LAF），通过变分表示分布近似与对比损失实现无需标签的深度模型机器遗忘，并在多种任务上达到接近全监督方法的效果。
摘要：
机器遗忘旨在从训练良好的模型中移除源自被遗忘数据的信息，同时保留剩余数据集的信息。随着数据隐私日益受到重视，已经出现了若干机器遗忘方法。然而，这些方法通常在整个遗忘过程中依赖完整监督。遗憾的是，由于标注真实世界数据集需要高昂成本，无论是针对被遗忘数据还是剩余数据，获得此类监督都可能不切实际。这一挑战促使我们提出一种无监督遗忘方法，在遗忘过程中无需标签即可运行。具体而言，我们引入一种变分方法来近似剩余数据的表示分布。利用该近似，我们调整原始模型，以在表示层面消除来自被遗忘数据的信息。为进一步解决缺乏监督信息、从而阻碍与真实标签对齐的问题，我们引入对比损失，以促进剩余数据表示与原始模型表示之间的匹配，从而保持预测性能。在多种遗忘任务上的实验结果证明了我们提出的方法 Label-Agnostic Forgetting（LAF，标签无关遗忘）在不使用任何标签情况下的有效性，其性能可与依赖完整监督信息的最先进方法相媲美。此外，我们的方法在半监督场景中表现出色，能够利用有限监督信息超越全监督基线。本工作不仅展示了深度模型中无监督遗忘的可行性，也为未来在表示层面开展遗忘研究开辟了新的可能。
英文摘要：
Machine unlearning aims to remove information derived from forgotten data while preserving that of the remaining dataset in a well-trained model. With the increasing emphasis on data privacy, several approaches to machine unlearning have emerged. However, these methods typically rely on complete supervision throughout the unlearning process. Unfortunately, obtaining such supervision, whether for the forgetting or remaining data, can be impractical due to the substantial cost associated with annotating real-world datasets. This challenge prompts us to propose a supervision-free unlearning approach that operates without the need for labels during the unlearning process. Specifically, we introduce a variational approach to approximate the distribution of representations for the remaining data. Leveraging this approximation, we adapt the original model to eliminate information from the forgotten data at the representation level. To further address the issue of lacking supervision information, which hinders alignment with ground truth, we introduce a contrastive loss to facilitate the matching of representations between the remaining data and those of the original model, thus preserving predictive performance. Experimental results across various unlearning tasks demonstrate the effectiveness of our proposed method, Label-Agnostic Forgetting (LAF) without using any labels, which achieves comparable performance to state-of-the-art methods that rely on full supervision information. Furthermore, our approach excels in semi-supervised scenarios, leveraging limited supervision information to outperform fully supervised baselines. This work not only showcases the viability of supervision-free unlearning in deep models but also opens up a new possibility for future research in unlearning at the representation level.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
