arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2106.15093cs.LGcs.DB
收藏

线性模型的可证明机器遗忘

Certifiable Machine Unlearning for Linear Models

University of Helsinki(赫尔辛基大学)

Ananth Mahadevan, Michael Mathioudakis

摘要:

机器遗忘是在删除训练数据的子集之后更新机器学习(ML)模型的任务。期望用于该任务的方法结合有效性和效率,即,他们应该有效地“忘记”已删除的数据,但以不需要过多计算工作的方式(例如,完全再训练)用于少量删除。这样的组合通常是通过在遗忘中容忍一定量的近似来实现的。此外,本着“被遗忘权”的精神,法律和条例提出了可证明性要求,即,能够证明被删除的数据确实已经被ML模型遗忘。在本文中,我们提出了一个实验研究的三个国家的最先进的近似unlearning方法的线性模型,并证明了效率之间的权衡,有效性和可证明性提供的每种方法。在实施这项研究的过程中,我们扩展了一些现有的工作,并描述了一个通用的ML管道,以比较和评估六个真实世界数据集和各种设置上的非学习方法。我们提供了关于删除数据的数量和分布对ML模型的影响以及不同设置中每种学习方法的性能的见解。我们还提出了一种实用的在线策略,以确定近似学习的累积误差何时足够大,以保证ML模型的完全重新训练。

英文摘要:

Machine unlearning is the task of updating machine learning (ML) models after a subset of the training data they were trained on is deleted. Methods for the task are desired to combine effectiveness and efficiency, i.e., they should effectively "unlearn" deleted data, but in a way that does not require excessive computation effort (e.g., a full retraining) for a small amount of deletions. Such a combination is typically achieved by tolerating some amount of approximation in the unlearning. In addition, laws and regulations in the spirit of "the right to be forgotten" have given rise to requirements for certifiability, i.e., the ability to demonstrate that the deleted data has indeed been unlearned by the ML model. In this paper, we present an experimental study of the three state-of-the-art approximate unlearning methods for linear models and demonstrate the trade-offs between efficiency, effectiveness and certifiability offered by each method. In implementing the study, we extend some of the existing works and describe a common ML pipeline to compare and evaluate the unlearning methods on six real-world datasets and a variety of settings. We provide insights into the effect of the quantity and distribution of the deleted data on ML models and the performance of each unlearning method in different settings. We also propose a practical online strategy to determine when the accumulated error from approximate unlearning is large enough to warrant a full retrain of the ML model.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑