arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2106.04378cs.LGstat.ML
收藏

自适应机器遗忘

Adaptive Machine Unlearning

University of Pennsylvania(宾夕法尼亚大学);Harvard University(哈佛大学);Stanford University(斯坦福大学)

Varun Gupta, Christopher Jung, Seth Neel, Aaron Roth, Saeed Sharifi-Malvajerdi, Chris Waites

摘要:

数据删除算法旨在以比完全重新训练这些模型更便宜的计算成本从训练模型中消除删除数据点的影响。然而,对于序列的删除,大多数以前的工作在非凸设置提供了有效的保证,只有序列的选择独立的模型,已公布的。如果人们选择根据发布的模型删除数据(例如,因为他们不喜欢模型显示的关于他们的信息),那么更新序列是自适应的。本文利用差分隐私及其与最大信息量的关系,给出了从自适应序列的删除保证到非自适应序列的删除保证的一般约简.结合以前的工作,保证非自适应删除序列的想法,这导致非常灵活的算法能够处理任意的模型类和训练方法,提供强有力的可证明的删除保证自适应删除序列。我们在理论上展示了非凸模型的先前工作如何在自适应删除序列上失败,并使用这种直觉来设计针对Bourtoule等人的SISA算法的实际攻击。[2021] CIFAR-10,MNIST,Fashion-MNIST。

英文摘要:

Data deletion algorithms aim to remove the influence of deleted data points from trained models at a cheaper computational cost than fully retraining those models. However, for sequences of deletions, most prior work in the non-convex setting gives valid guarantees only for sequences that are chosen independently of the models that are published. If people choose to delete their data as a function of the published models (because they don't like what the models reveal about them, for example), then the update sequence is adaptive. In this paper, we give a general reduction from deletion guarantees against adaptive sequences to deletion guarantees against non-adaptive sequences, using differential privacy and its connection to max information. Combined with ideas from prior work which give guarantees for non-adaptive deletion sequences, this leads to extremely flexible algorithms able to handle arbitrary model classes and training methodologies, giving strong provable deletion guarantees for adaptive deletion sequences. We show in theory how prior work for non-convex models fails against adaptive deletion sequences, and use this intuition to design a practical attack against the SISA algorithm of Bourtoule et al. [2021] on CIFAR-10, MNIST, Fashion-MNIST.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑