arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2204.07655cs.CVcs.LG
收藏

基于随机卷积无关Hessian的深度非学习

Deep Unlearning via Randomized Conditionally Independent Hessians

University of Wisconsin-Madison(威斯康星大学麦迪逊分校);University of Illinois at Chicago(伊利诺伊大学芝加哥分校)

Ronak Mehta, Sourav Pal, Vikas Singh, Sathya N. Ravi

摘要:

最近的立法已经引起了对机器遗忘的兴趣,即,从预测模型中删除特定的训练样本,就好像它们从未存在于训练数据集中一样。由于损坏的/对抗性的数据或仅仅是用户更新的隐私要求,也可能需要遗忘。对于不需要训练的模型(k-NN),简单地删除最接近的原始样本可能是有效的。但是这个想法不适用于学习更丰富表示的模型。利用基于优化的更新的最新想法与模型维度d的比例很差,这是由于反转了损失函数的Hessian。我们使用一个新的条件独立系数,L-CODEC的变体,以确定一个子集的模型参数与最语义重叠的个人样本水平。我们的方法完全避免了需要反转(可能)巨大的矩阵。通过利用马尔可夫毯选择,我们假设L-CODEC也适用于深度学习以及视觉中的其他应用。与替代方案相比,L-CODEC使近似遗忘在其他不可行的设置中成为可能,包括用于人脸识别的视觉模型,人员重新识别和可能需要识别排除的遗忘样本的NLP模型。代码可以在https://github.com/vsingh-group/LCODEC-deep-unlearning/上找到

英文摘要:

Recent legislation has led to interest in machine unlearning, i.e., removing specific training samples from a predictive model as if they never existed in the training dataset. Unlearning may also be required due to corrupted/adversarial data or simply a user's updated privacy requirement. For models which require no training (k-NN), simply deleting the closest original sample can be effective. But this idea is inapplicable to models which learn richer representations. Recent ideas leveraging optimization-based updates scale poorly with the model dimension d, due to inverting the Hessian of the loss function. We use a variant of a new conditional independence coefficient, L-CODEC, to identify a subset of the model parameters with the most semantic overlap on an individual sample level. Our approach completely avoids the need to invert a (possibly) huge matrix. By utilizing a Markov blanket selection, we premise that L-CODEC is also suitable for deep unlearning, as well as other applications in vision. Compared to alternatives, L-CODEC makes approximate unlearning possible in settings that would otherwise be infeasible, including vision models used for face recognition, person re-identification and NLP models that may require unlearning samples identified for exclusion. Code can be found at https://github.com/vsingh-group/LCODEC-deep-unlearning/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑