arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2201.05629cs.LGcs.AI
收藏

零样本机器遗忘

Zero-Shot Machine Unlearning

Mavvex Labs(Mavvex实验室);Kalinga Institute of Industrial Technology(卡林加工业技术学院);National University of Singapore(新加坡国立大学)

Vikram S Chundawat, Ayush K Tarun, Murari Mandal, Mohan Kankanhalli

AI总结:

本文提出零样本机器遗忘问题,在无任何原始训练样本可用的情况下,通过误差最小化-最大化噪声和门控知识迁移两种方法移除模型中特定数据信息,并引入记忆指数(AIN)评估遗忘质量,实验验证了其有效性。

摘要:

现代隐私法规赋予公民被产品、服务和公司遗忘的权利。在机器学习应用中,这不仅要求从存储档案中删除数据,还要求从机器学习模型中删除数据。由于机器学习应用对法规合规性的需求日益增长,机器遗忘正在成为一个新兴的研究问题。被遗忘权请求的形式是从已训练的机器学习模型中移除某一特定集合或类别的数据。实际考虑排除了在丢弃被删除数据后从头重新训练模型的做法。现有的少数研究要么使用全部训练数据,要么使用训练数据的子集,要么使用训练期间存储的某些元数据来更新模型权重以实现遗忘。然而,在许多情况下,可能无法访问与训练过程或训练样本相关的任何数据来进行遗忘。因此我们提出一个问题:是否可能在零训练样本的情况下实现遗忘?在本文中,我们引入了零样本机器遗忘这一新问题,以应对零原始数据样本可用的极端但实际的场景。随后,我们提出了两种基于(a)误差最小化-最大化噪声和(b)门控知识迁移的零样本机器遗忘新方案。这些方法从模型中移除遗忘数据的信息,同时保持模型在保留数据上的效能。零样本方法对模型反演攻击和成员推断攻击提供了良好的防护。我们引入了一个新的评估指标——记忆指数(AIN),以有效衡量遗忘方法的质量。实验表明,在基准视觉数据集上的深度学习模型中,遗忘取得了有前景的结果。源代码可在以下地址获取:https://github.com/ayu987/zero-shot-unlearning

英文摘要:

Modern privacy regulations grant citizens the right to be forgotten by products, services and companies. In case of machine learning (ML) applications, this necessitates deletion of data not only from storage archives but also from ML models. Due to an increasing need for regulatory compliance required for ML applications, machine unlearning is becoming an emerging research problem. The right to be forgotten requests come in the form of removal of a certain set or class of data from the already trained ML model. Practical considerations preclude retraining of the model from scratch after discarding the deleted data. The few existing studies use either the whole training data, or a subset of training data, or some metadata stored during training to update the model weights for unlearning. However, in many cases, no data related to the training process or training samples may be accessible for the unlearning purpose. We therefore ask the question: is it possible to achieve unlearning with zero training samples? In this paper, we introduce the novel problem of zero-shot machine unlearning that caters for the extreme but practical scenario where zero original data samples are available for use. We then propose two novel solutions for zero-shot machine unlearning based on (a) error minimizing-maximizing noise and (b) gated knowledge transfer. These methods remove the information of the forget data from the model while maintaining the model efficacy on the retain data. The zero-shot approach offers good protection against the model inversion attacks and membership inference attacks. We introduce a new evaluation metric, Anamnesis Index (AIN) to effectively measure the quality of the unlearning method. The experiments show promising results for unlearning in deep learning models on benchmark vision data-sets. The source code is available here: https://github.com/ayu987/zero-shot-unlearning

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑