arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2312.04095cs.LGcs.CV
收藏

面向深度神经网络的学会遗忘:利用梯度投影最小化遗忘干扰

Learn to Unlearn for Deep Neural Networks: Minimizing Unlearning Interference with Gradient Projection

Deakin University(迪肯大学)

Tuan Hoang, Santu Rana, Sunil Gupta, Svetha Venkatesh

AI总结:

针对机器遗忘需删除指定样本影响且保留剩余知识的问题,提出 PGU 方法,通过正交于保留数据梯度子空间的投影梯度与 SGD 更新实现高效、可扩展遗忘。

摘要:

近期的数据隐私法规引发了人们对机器遗忘(machine unlearning)的兴趣,该任务涉及从已学习模型中移除特定训练样本的影响,使其如同从未出现在原始训练数据集中一样。机器遗忘的挑战在于:在丢弃已学习模型中关于“遗忘”数据的信息的同时,不改变关于剩余数据集的知识,并且要比朴素的重新训练方法更高效。为实现这一目标,我们采用了一种基于投影梯度的学习方法,命名为 Projected-Gradient Unlearning(PGU,投影梯度遗忘);在该方法中,模型沿与被认为对保留数据集不重要的梯度子空间正交的方向迈步,从而使其知识得到保留。通过利用随机梯度下降(SGD)更新模型权重,我们的方法能够高效扩展至任意模型和数据集规模。我们提供了实验证据,证明即使在无法再访问训练数据集的情况下,我们的遗忘方法也能生成在多项指标上表现与从头重新训练的模型相似的模型。我们的代码可在 https://github.com/hnanhtuan/projected_gradient_unlearning 获取。

英文摘要:

Recent data-privacy laws have sparked interest in machine unlearning, which involves removing the effect of specific training samples from a learnt model as if they were never present in the original training dataset. The challenge of machine unlearning is to discard information about the ``forget'' data in the learnt model without altering the knowledge about the remaining dataset and to do so more efficiently than the naive retraining approach. To achieve this, we adopt a projected-gradient based learning method, named as Projected-Gradient Unlearning (PGU), in which the model takes steps in the orthogonal direction to the gradient subspaces deemed unimportant for the retaining dataset, so as to its knowledge is preserved. By utilizing Stochastic Gradient Descent (SGD) to update the model weights, our method can efficiently scale to any model and dataset size. We provide empirically evidence to demonstrate that our unlearning method can produce models that behave similar to models retrained from scratch across various metrics even when the training dataset is no longer accessible. Our code is available at https://github.com/hnanhtuan/projected_gradient_unlearning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑