机器遗忘已成为实现“被遗忘权”的一种有前景的解决方案,在该权利下,个人可以请求从机器学习模型中删除其数据。然而,现有机器遗忘研究主要关注遗忘方法的有效性和效率,却忽视了对遗忘过程中隐私漏洞的考察。当攻击者能够获得一个模型的两个版本,即原始模型和遗忘后模型时,机器遗忘便开启了一个新的攻击面。在本文中,我们首次开展研究,以理解机器遗忘在多大程度上会泄露被遗忘数据的机密内容。具体而言,在 Machine Learning as a Service(机器学习即服务)设置下,我们提出了遗忘反演攻击,该攻击仅通过访问原始模型和遗忘后模型,就能够揭示被遗忘样本的特征和标签信息。我们通过在基准数据集上、跨多种模型架构,并针对精确和近似两类代表性遗忘方法开展大量实验,评估了所提出遗忘反演攻击的有效性。实验结果表明,所提出的攻击能够揭示被遗忘数据的敏感信息。因此,我们识别出三种可能的防御措施,它们有助于缓解所提出的攻击,但代价是降低遗忘后模型的效用。本文研究揭示了机器遗忘与被遗忘数据隐私之间一个尚未得到充分探索的缺口,强调需要谨慎设计相关机制,以在实现遗忘的同时不泄露被遗忘数据的信息。
英文摘要:
Machine unlearning has become a promising solution for fulfilling the "right to be forgotten", under which individuals can request the deletion of their data from machine learning models. However, existing studies of machine unlearning mainly focus on the efficacy and efficiency of unlearning methods, while neglecting the investigation of the privacy vulnerability during the unlearning process. With two versions of a model available to an adversary, that is, the original model and the unlearned model, machine unlearning opens up a new attack surface. In this paper, we conduct the first investigation to understand the extent to which machine unlearning can leak the confidential content of the unlearned data. Specifically, under the Machine Learning as a Service setting, we propose unlearning inversion attacks that can reveal the feature and label information of an unlearned sample by only accessing the original and unlearned model. The effectiveness of the proposed unlearning inversion attacks is evaluated through extensive experiments on benchmark datasets across various model architectures and on both exact and approximate representative unlearning approaches. The experimental results indicate that the proposed attack can reveal the sensitive information of the unlearned data. As such, we identify three possible defenses that help to mitigate the proposed attacks, while at the cost of reducing the utility of the unlearned model. The study in this paper uncovers an underexplored gap between machine unlearning and the privacy of unlearned data, highlighting the need for the careful design of mechanisms for implementing unlearning without leaking the information of the unlearned data.
Zero-shot Class Unlearning via Layer-wise Relevance Analysis and Neuronal Path Perturbation
School of Information Engineering, Zhongnan University of Economics and Law(信息工程学院,中南财经政法大学);China University of Geosciences, Wuhan(中国地质大学(武汉));City University of Macau(澳门城市大学)
Machine Unlearning with Minimal Gradient Dependence for High Unlearning Ratios
School of Information, Renmin University of China(中国人民大学信息学院);School of Computer Science and Big Data, Minjiang University(闽江学院计算机与大数据学院);RMIT University(皇家墨尔本理工大学)
Towards Lifecycle Unlearning Commitment Management: Measuring Sample-level Unlearning Completeness
Provable Responsible AI and Data Analytics Lab(可证负责任人工智能与数据 analytics 实验室);King Abdullah University of Science and Technology(卡塔尔国王 Abdullah 科学与技术大学);National University of Singapore(新加坡国家大学);Johns Hopkins University(约翰霍普金斯大学)
本文提出 IAM 框架,通过插值模型在样本上的泛化—拟合差距度量样本级遗忘完整性,可低成本扩展至 LLM,并揭示近似遗忘中过度与不足遗忘风险。
TruVRF: Towards Triple-Granularity Verification on Machine Unlearning
School of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);School of Computer Science and Engineering, Nanjing University of Science and Technology(南京理工大学计算机科学与工程学院)
Towards Irreversible Machine Unlearning for Diffusion Models
Department of Electrical and Computer Engineering, College of Design and Engineering, National University of Singapore(新加坡国立大学电子与计算机工程系,设计与工程学院);Betterdata, Singapore(新加坡Betterdata公司);Department of Computer Science, University of Sheffield(谢菲尔德大学计算机科学系)
Textual Unlearning Gives a False Sense of Unlearning
The State Key Laboratory of Blockchain and Data Security, Zhejiang University, P. R. China(区块链与数据安全国家重点实验室,浙江大学,中华人民共和国);School of Cyber Science and Technology, Zhejiang University, P. R. China(网络安全与技术学院,浙江大学,中华人民共和国);ETH Zurich, Switzerland(苏黎世联邦理工学院,瑞士)
BlindU: Blind Machine Unlearning without Revealing Erasing Data
School of Computer Science, University of Technology Sydney(计算机科学学院,技术悉尼大学);School of Cyber Science and Engineering, Southeast University(网络科学与工程学院,东南大学)
Towards Robust Knowledge Unlearning: An Adversarial Framework for Assessing and Improving Unlearning Robustness in Large Language Models
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
Poisoning Attacks and Defenses to Federated Unlearning
ShanghaiTech University(上海科技大学);Xidian University(西安电子科技大学);North Carolina State University(北卡罗来纳州立大学);University of North Texas(北德克萨斯大学);University of Louisville(路易斯维尔大学)