arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.09591cs.CR
收藏

Unlearn and Burn:对抗性机器遗忘请求摧毁模型准确率

Unlearn and Burn: Adversarial Machine Unlearning Requests Destroy Model Accuracy

Yangsibo Huang, Daogao Liu, Lynn Chua, Badih Ghazi, Pritish Kamath, Ravi Kumar, Pasin Manurangsi, Milad Nasr, Amer Sinha, Chiyuan Zhang

AI总结:

本研究揭示了机器遗忘系统假设移除数据必在原训练集中的漏洞,提出白盒与黑盒对抗性遗忘请求攻击算法,在CIFAR-10和ImageNet上使模型准确率骤降,并指出当前验证机制难以检测隐蔽攻击。

摘要:

机器遗忘算法旨在从模型中选择性移除训练数据,已成为应对日益增长的隐私问题的有前景的方法。在本研究中,我们揭示了遗忘系统部署中一个关键但尚未充分探索的漏洞:即假设请求移除的数据始终是原始训练集的一部分。我们提出了一种威胁模型,攻击者可以通过提交针对训练集中不存在的数据的对抗性遗忘请求来降低模型准确率。我们提出了白盒和黑盒攻击算法,并通过在CIFAR-10和ImageNet数据集上的图像分类任务案例研究对其进行了评估,目标是针对一系列广泛使用的遗忘方法。结果显示,攻击后的测试准确率极低:白盒攻击在CIFAR-10上为3.6%,在ImageNet上为0.4%;黑盒攻击在CIFAR-10上为8.5%,在ImageNet上为1.3%。此外,我们评估了各种验证机制以检测遗忘请求的合法性,并揭示了验证中的挑战,因为大多数机制在不严重损害其处理有效请求能力的情况下,无法检测到隐蔽攻击。这些发现强调,如果未来机器遗忘系统的部署变得更加普遍,迫切需要研究更稳健的请求验证方法和遗忘协议。

英文摘要:

Machine unlearning algorithms, designed for selective removal of training data from models, have emerged as a promising approach to growing privacy concerns. In this work, we expose a critical yet underexplored vulnerability in the deployment of unlearning systems: the assumption that the data requested for removal is always part of the original training set. We present a threat model where an attacker can degrade model accuracy by submitting adversarial unlearning requests for data not present in the training set. We propose white-box and black-box attack algorithms and evaluate them through a case study on image classification tasks using the CIFAR-10 and ImageNet datasets, targeting a family of widely used unlearning methods. Our results show extremely poor test accuracy following the attack: 3.6% on CIFAR-10 and 0.4% on ImageNet for white-box attacks, and 8.5% on CIFAR-10 and 1.3% on ImageNet for black-box attacks. Additionally, we evaluate various verification mechanisms to detect the legitimacy of unlearning requests and reveal the challenges in verification, as most of the mechanisms fail to detect stealthy attacks without severely impairing their ability to process valid requests. These findings underscore the urgent need for research on more robust request verification methods and unlearning protocols, should the deployment of machine unlearning systems become more prevalent in the future.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑