We propose new methodologies for both unlearning random set of samples and class unlearning and show that they outperform existing methods. The main driver of our unlearning methods is the similarity of predictions to a retrained model on both the forget and remain samples. We introduce Adversarial Machine UNlearning (AMUN), which surpasses prior state-of-the-art methods for image classification based on SOTA MIA scores. AMUN lowers the model's confidence on forget samples by fine-tuning on their corresponding adversarial examples. Through theoretical analysis, we identify factors governing AMUN's performance, including smoothness. To facilitate training of smooth models with a controlled Lipschitz constant, we propose FastClip, a scalable method that performs layer-wise spectral-norm clipping of affine layers. In a separate study, we show that increased smoothness naturally improves adversarial example transfer, thereby supporting the second factor above.
Following the same principles for class unlearning, we show that existing methods fail in replicating a retrained model's behavior by introducing a nearest-neighbor membership inference attack (MIA-NN) that uses the probabilities assigned to neighboring classes to detect unlearned samples and demonstrate the vulnerability of such methods. We then propose a fine-tuning objective that mitigates this leakage by approximating, for forget-class inputs, the distribution over remaining classes that a model retrained from scratch would produce. To construct this approximation, we estimate inter-class similarity and tilt the target model's distribution accordingly. The resulting Tilted ReWeighting(TRW) distribution serves as the desired target during fine-tuning. Across multiple benchmarks, TRW matches or surpasses existing unlearning methods on prior metrics.
Langevin Unlearning: A New Perspective of Noisy Gradient Descent for Machine Unlearning
Department of Electrical and Computer Engineering(电气与计算机工程系);Georgia Institute of Technology(佐治亚理工学院);Department of Mathematics(数学系);Massachusetts Institute of Technology(麻省理工学院)
Michigan State University(密歇根州立大学);IBM Research(IBM研究院);Cisco Research(思科研发);University of California, Santa Cruz(加州大学圣克鲁兹分校);ByteDance Research(字节跳动研究);Carnegie Mellon University(卡内基梅隆大学)
Protecting the Neural Networks against FGSM Attack Using Machine Unlearning
Department of Computer Engineering, Faculty of Engineering, Saveh Branch, Islamic Azad University(伊斯兰阿扎德大学萨韦分校工程学院计算机工程系);Faculty of Computer Science and Engineering, Shahid Beheshti University(沙希德·贝赫什提大学计算机科学与工程学院)
BIFOLD & TU Berlin(BIFOLD与柏林工业大学);CISPA Helmholtz Center for Information Security(CISPA亥姆霍兹信息安全中心);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Microsoft Research(微软研究院);The University of Melbourne(墨尔本大学);University of Toronto & Vector Institute(多伦多大学与向量研究所)
MeGU: Machine-Guided Unlearning with Target Feature Disentanglement
School of Automation, Beijing Institute of Technology(自动化学院,北京理工大学);Sydney AI Centre, The University of Sydney(悉尼人工智能中心,悉尼大学);Department of Computer Science, Hong Kong Baptist University(计算机科学系,香港 Baptist 大学)
Machine unlearning through fine-grained model parameters perturbation
College of Computer Science and Electronic Engineering, Hunan University(湖南大学计算机与电子工程学院);College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院);Shenzhen Research Institute, Hunan University(湖南大学深圳研究院);Faculty of Computing, Engineering & Media, De Montfort University(德蒙福特大学计算、工程与媒体学院)