arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.09880cs.LGcs.CR
收藏

迈向无界机器遗忘

Towards Unbounded Machine Unlearning

University of Warwick(华威大学);Google DeepMind(谷歌DeepMind)

Meghdad Kurmanji, Peter Triantafillou, Jamie Hayes, Eleni Triantafillou

AI总结:

本文首次系统研究面向移除偏见、解决混淆和用户隐私三类应用的深度机器遗忘问题,提出新颖的SCRUB遗忘算法,在遗忘质量与模型效用上均持续领先且更高效。

摘要:

深度机器遗忘是指从已训练的神经网络中“移除”其训练集的一个子集的问题。该问题非常具有时效性,并拥有众多应用,包括移除偏见(RB)、解决混淆(RC)(由训练模型中错误标注的数据引起)的关键任务,以及允许用户行使“被遗忘权”以保护用户隐私(UP)。据我们所知,本文是首个针对不同应用(RB、RC、UP)研究遗忘的工作,并认为每种应用都有其自身的设计目标、“遗忘”的定义以及相关的遗忘质量度量标准。针对UP,我们提出了一种针对遗忘的强成员推断攻击的新颖改编方法。我们还提出了SCRUB,一种新颖的遗忘算法,它是唯一一种在RB、RC和UP的不同应用相关度量上持续在遗忘质量方面表现最优的方法。同时,SCRUB在衡量模型效用(即在保留数据上的准确性和泛化能力)的指标上也持续表现最优,并且比先前的工作更高效。上述结论通过对先前最先进方法的全面实证评估得到了证实。

英文摘要:

Deep machine unlearning is the problem of `removing' from a trained neural network a subset of its training set. This problem is very timely and has many applications, including the key tasks of removing biases (RB), resolving confusion (RC) (caused by mislabelled data in trained models), as well as allowing users to exercise their `right to be forgotten' to protect User Privacy (UP). This paper is the first, to our knowledge, to study unlearning for different applications (RB, RC, UP), with the view that each has its own desiderata, definitions for `forgetting' and associated metrics for forget quality. For UP, we propose a novel adaptation of a strong Membership Inference Attack for unlearning. We also propose SCRUB, a novel unlearning algorithm, which is the only method that is consistently a top performer for forget quality across the different application-dependent metrics for RB, RC, and UP. At the same time, SCRUB is also consistently a top performer on metrics that measure model utility (i.e. accuracy on retained data and generalization), and is more efficient than previous work. The above are substantiated through a comprehensive empirical evaluation against previous state-of-the-art.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑