Removing information from a machine learning model is a non-trivial task that requires to partially revert the training process. This task is unavoidable when sensitive data, such as credit card numbers or passwords, accidentally enter the model and need to be removed afterwards. Recently, different concepts for machine unlearning have been proposed to address this problem. While these approaches are effective in removing individual data points, they do not scale to scenarios where larger groups of features and labels need to be reverted. In this paper, we propose the first method for unlearning features and labels. Our approach builds on the concept of influence functions and realizes unlearning through closed-form updates of model parameters. It enables to adapt the influence of training data on a learning model retrospectively, thereby correcting data leaks and privacy issues. For learning models with strongly convex loss functions, our method provides certified unlearning with theoretical guarantees. For models with non-convex losses, we empirically show that unlearning features and labels is effective and significantly faster than other strategies.
Certifying the Right to Be Forgotten: Primal-Dual Optimization for Sample and Label Unlearning in Vertical Federated Learning
College of Computing and Data Science (CCDS), Nanyang Technological University, Singapore and Digital Trust Centre (DTC), Singapore(南洋理工大学计算与数据科学学院(CCDS)及新加坡数字信任中心(DTC));College of Computing and Data Science (CCDS), Nanyang Technological University, Singapore(南洋理工大学计算与数据科学学院(CCDS));School of Computer Science and Engineering, Sun Yat-sen University(中山大学计算机科学与工程学院)
Zero-shot Class Unlearning via Layer-wise Relevance Analysis and Neuronal Path Perturbation
School of Information Engineering, Zhongnan University of Economics and Law(信息工程学院,中南财经政法大学);China University of Geosciences, Wuhan(中国地质大学(武汉));City University of Macau(澳门城市大学)
TruVRF: Towards Triple-Granularity Verification on Machine Unlearning
School of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);School of Computer Science and Engineering, Nanjing University of Science and Technology(南京理工大学计算机科学与工程学院)
Towards Efficient Target-Level Machine Unlearning Based on Essential Graph
University of Technology Sydney(悉尼科技大学);City University of Macau(澳门城市大学);Shenzhen Institute of Advanced Technology, University of Chinese Academy of Sciences(中国科学院大学深圳先进技术研究院)
Machine Unlearning with Minimal Gradient Dependence for High Unlearning Ratios
School of Information, Renmin University of China(中国人民大学信息学院);School of Computer Science and Big Data, Minjiang University(闽江学院计算机与大数据学院);RMIT University(皇家墨尔本理工大学)
Langevin Unlearning: A New Perspective of Noisy Gradient Descent for Machine Unlearning
Department of Electrical and Computer Engineering(电气与计算机工程系);Georgia Institute of Technology(佐治亚理工学院);Department of Mathematics(数学系);Massachusetts Institute of Technology(麻省理工学院)
Machine Unlearning: Taxonomy, Metrics, Applications, Challenges, and Prospects
Nanjing University of Science and Technology(南京理工大学);Xidian University(西安电子科技大学);CSIRO(联邦科学与工业研究组织);University of Western Australia(西澳大学);University of Technology Sydney(悉尼科技大学)
Federated Unlearning Model Recovery in Data with Skewed Label Distributions
School of Computer Science and Technology(计算机科学与技术学院);Dalian University of Technology(大连理工大学);School of Engineering and Computer Science(工程与计算机科学学院);Victoria University of Wellington(惠灵顿维多利亚大学)
University of Virginia(弗吉尼亚大学);University of Lausanne(洛桑大学);CISPA Helmholtz Center for Information Security(CISPA亥姆霍兹信息安全中心);Carnegie Mellon University(卡内基梅隆大学);Purdue University(普渡大学)
Machine unlearning through fine-grained model parameters perturbation
College of Computer Science and Electronic Engineering, Hunan University(湖南大学计算机与电子工程学院);College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院);Shenzhen Research Institute, Hunan University(湖南大学深圳研究院);Faculty of Computing, Engineering & Media, De Montfort University(德蒙福特大学计算、工程与媒体学院)
Michigan State University(密歇根州立大学);IBM Research(IBM研究院);Cisco Research(思科研发);University of California, Santa Cruz(加州大学圣克鲁兹分校);ByteDance Research(字节跳动研究);Carnegie Mellon University(卡内基梅隆大学)
Machine Unlearning of Pre-trained Large Language Models
University of Virginia(弗吉尼亚大学);Georgia Institute of Technology(佐治亚理工学院);The Hong Kong Polytechnic University(香港理工大学);University of Melbourne(墨尔本大学);Carnegie Mellon University(卡内基梅隆大学)