Machine unlearning is an emerging technology that removes a subset of the training data from a trained model without significantly affecting the model performance on the remaining data. This topic is becoming increasingly important in protecting user privacy and eliminating harmful or outdated data. The key challenge lies in effectively and efficiently unlearning specific information without compromising the model's utility on the retained data. For pre-trained models, fine-tuning is an important way to achieve the unlearning target. Previous work typically fine-tuned the entire model's parameters, which incurred significant computational costs. In addition, the fine-tuning process may cause shifts in the intermediate layer features, affecting the model's overall utility. In this work, we propose a novel and efficient machine unlearning method for pre-trained models. We term the method Residual Feature Alignment Unlearning. Specifically, we leverage LoRA (Low-Rank Adaptation) to decompose the model's intermediate features into pre-trained features and residual features. By adjusting the residual features, we align the unlearned model with the pre-trained model at the intermediate feature level to achieve both unlearning and remaining targets. The method aims to learn zero residuals on the retained set and shifted residuals on the unlearning set. Extensive experiments on numerous datasets validate the effectiveness of our approach.
Orthogonal Subspace Projection for Continual Machine Unlearning via SVD-Based LoRA
Institute for Digital Technologies, Loughborough University, UK(伦敦大学数字技术研究所,英国);School of Artificial Intelligent Medicine, Guilin Medical University(桂林医学院人工智能医学院)
Auditing Language Model Unlearning via Information Decomposition
Technical University of Darmstadt(达姆施塔特工业大学);National Research Center for Applied Cybersecurity ATHENE(ATHENE国家应用网络安全研究中心);Georgia Institute of Technology(佐治亚理工学院)
DeepObliviate: A Powerful Charm for Erasing Data Residual Memory in Deep Neural Networks
SKLOIS, Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所智能信息处理实验室);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
U-CAN: Utility-Aware Contrastive Attenuation for Efficient Unlearning in Generative Recommendation
Guilin University of Electronic Technology(桂林电子科技大学);University of Southampton(南安普顿大学);Jinan University(暨南大学);Nanjing University of Science and Technology(南京理工大学)
MeGU: Machine-Guided Unlearning with Target Feature Disentanglement
School of Automation, Beijing Institute of Technology(自动化学院,北京理工大学);Sydney AI Centre, The University of Sydney(悉尼人工智能中心,悉尼大学);Department of Computer Science, Hong Kong Baptist University(计算机科学系,香港 Baptist 大学)
Indian Institute of Information Technology, Bhopal, India(印度比哈尔理工学院);University of California, Santa Cruz, USA(加州大学圣克鲁兹分校);Independent Researcher(独立研究者);Stanford University, USA(斯坦福大学);BITS Pilani Goa, India(比斯拉米印度学院)
Certified Unlearning in Decentralized Federated Learning
School of Computer Science and Technology, Shandong University, Qingdao, China(山东大学计算机科学与技术学院);School of Electrical Engineering and Computer Science, TU Berlin, Berlin, Germany(柏林技术大学电气工程与计算机科学学院);College of Computer Science and Technology, Ocean University of China, Qingdao, China(中国海洋大学计算机科学与技术学院)
Inference-Time Machine Unlearning via Gated Activation Redirection
MALTA, Machine Learning Theory and Applications Lab, PUCRS, Porto Alegre, Brazil(MALTA机器学习理论与应用实验室,PUCRS,波士顿-阿尔格雷,巴西);Harvard University(哈佛大学);Kunumi Institute, Brazil(库努米研究所,巴西)
Towards Probabilistic Verification of Machine Unlearning
ETH Zürich(苏黎世联邦理工学院);Princeton University(普林斯顿大学)
被遗忘权(Right to be forgotten),也被称为删除权(Right to erasure),是指个人有权将其数据从存储数据的实体中删除。这一长期持有的概念的地位最近在欧盟的《通用数据保护条例》(GDPR)中得到了法律上的巩固。因此,有必要建立一种机制,使用户能够核实服务提供商是否遵守其删除请求。在这项工作中,我们采取的第一步,提出了一个正式的框架,研究设计的验证机制,数据删除请求-也被称为机器遗忘-的背景下,提供机器学习作为一种服务(MLaaS)的系统。我们的框架允许严格量化的任何验证机制的基础上,标准的假设检验。此外,我们提出了一种新的基于后门的验证机制,并证明其有效性,在证明数据删除具有高置信度,从而提供了一个基础,定量推断机器遗忘。我们在一系列网络架构上评估了我们的方法,例如多层感知器(MLP),卷积神经网络(CNN),残差网络(ResNet)和长短期记忆(LSTM),以及超过5个不同的数据集。我们证明了我们的方法对ML服务的准确性影响最小,但提供了高置信度的遗忘验证。即使只有少数用户使用我们的系统来确定是否符合删除数据的要求,我们建议的机制也可以工作。特别是,只有5%的用户参与,用后门修改了一半的数据,只有30个测试查询,我们的验证机制的误报率和漏报率都低于$10^{-3}$。我们还显示了我们的方法的有效性,通过测试它对一个自适应的对手,使用一个国家的最先进…
Priority-Aware Learning-Unlearning Correction for Dynamic Decentralized LoRA Fine-Tuning
Beijing Key Laboratory of Network System Architecture and Convergence, Beijing University of Posts and Telecommunications(网络系统架构与融合北京市重点实验室,北京邮电大学);Information Systems Technology and Design Pillar, Singapore University of Technology and Design(新加坡科技设计大学信息系统技术与设计系)
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Zhejiang University(浙江大学);University of Sydney(悉尼大学);Northwestern University(西北大学);Analogy AI, Inc.(Analogy AI公司);Mohamed Bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)