针对机器遗忘中遗忘与保留目标的梯度冲突问题，提出 Learning to Unlearn while Retaining，通过隐式梯度正则化兼顾有效遗忘与模型性能。
摘要：
机器遗忘最近引起了广泛关注，其目标是选择性地移除与特定数据相关的知识，同时保留模型在剩余数据上的性能。这一过程中的一个根本挑战是平衡有效遗忘与知识保留，因为对这些相互竞争目标的朴素优化可能导致梯度冲突，从而阻碍收敛并降低整体性能。为解决该问题，我们提出 Learning to Unlearn while Retaining，旨在缓解遗忘目标与保留目标之间的梯度冲突。我们的方法通过在所提出框架内自然产生的隐式梯度正则化机制，策略性地避免冲突。这防止了遗忘与保留之间的梯度冲突，从而在保留模型效用的同时实现有效遗忘。我们在判别式和生成式任务上验证了该方法，证明其能够在不损害剩余数据性能的情况下完成遗忘。结果突显了避免此类梯度冲突的优势，并且优于那些未考虑这些相互作用的现有方法。
英文摘要：
Machine Unlearning has recently garnered significant attention, aiming to selectively remove knowledge associated with specific data while preserving the model's performance on the remaining data. A fundamental challenge in this process is balancing effective unlearning with knowledge retention, as naive optimization of these competing objectives can lead to conflicting gradients, hindering convergence and degrading overall performance. To address this issue, we propose Learning to Unlearn while Retaining, aimed to mitigate gradient conflicts between unlearning and retention objectives. Our approach strategically avoids conflicts through an implicit gradient regularization mechanism that emerges naturally within the proposed framework. This prevents conflicting gradients between unlearning and retention, leading to effective unlearning while preserving the model's utility. We validate our approach across both discriminative and generative tasks, demonstrating its effectiveness in achieving unlearning without compromising performance on remaining data. Our results highlight the advantages of avoiding such gradient conflicts, outperforming existing methods that fail to account for these interactions.
Langevin Unlearning: A New Perspective of Noisy Gradient Descent for Machine Unlearning
Department of Electrical and Computer Engineering（电气与计算机工程系）；Georgia Institute of Technology（佐治亚理工学院）；Department of Mathematics（数学系）；Massachusetts Institute of Technology（麻省理工学院）
Pohang University of Science and Technology（浦项科技大学）
We consider a practical scenario of machine unlearning to erase a target dataset, which causes unexpected behavior from the trained model. The target dataset is often assumed to be fully identifiable in a standard unlearning scenario. Such a flawless identification, however, is almost impossible if the training dataset is inaccessible at the time of unlearning. Unlike previous approaches requiring a complete set of targets, we consider few-shot unlearning scenario when only a few samples of target data are available. To this end, we formulate the few-shot unlearning problem specifying intentio…
Michigan State University（密歇根州立大学）；IBM Research（IBM研究院）；Cisco Research（思科研发）；University of California, Santa Cruz（加州大学圣克鲁兹分校）；ByteDance Research（字节跳动研究）；Carnegie Mellon University（卡内基梅隆大学）
Zero-shot Class Unlearning via Layer-wise Relevance Analysis and Neuronal Path Perturbation
School of Information Engineering, Zhongnan University of Economics and Law（信息工程学院，中南财经政法大学）；China University of Geosciences, Wuhan（中国地质大学（武汉））；City University of Macau（澳门城市大学）
Machine Unlearning with Minimal Gradient Dependence for High Unlearning Ratios
School of Information, Renmin University of China（中国人民大学信息学院）；School of Computer Science and Big Data, Minjiang University（闽江学院计算机与大数据学院）；RMIT University（皇家墨尔本理工大学）
Towards Lifecycle Unlearning Commitment Management: Measuring Sample-level Unlearning Completeness
Provable Responsible AI and Data Analytics Lab（可证负责任人工智能与数据 analytics 实验室）；King Abdullah University of Science and Technology（卡塔尔国王 Abdullah 科学与技术大学）；National University of Singapore（新加坡国家大学）；Johns Hopkins University（约翰霍普金斯大学）
本文提出 IAM 框架，通过插值模型在样本上的泛化—拟合差距度量样本级遗忘完整性，可低成本扩展至 LLM，并揭示近似遗忘中过度与不足遗忘风险。
Textual Unlearning Gives a False Sense of Unlearning
The State Key Laboratory of Blockchain and Data Security, Zhejiang University, P. R. China（区块链与数据安全国家重点实验室，浙江大学，中华人民共和国）；School of Cyber Science and Technology, Zhejiang University, P. R. China（网络安全与技术学院，浙江大学，中华人民共和国）；ETH Zurich, Switzerland（苏黎世联邦理工学院，瑞士）
Machine unlearning through fine-grained model parameters perturbation
College of Computer Science and Electronic Engineering, Hunan University（湖南大学计算机与电子工程学院）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）；Shenzhen Research Institute, Hunan University（湖南大学深圳研究院）；Faculty of Computing, Engineering & Media, De Montfort University（德蒙福特大学计算、工程与媒体学院）
