arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2209.12269stat.MLcs.LG
收藏

近似数据删除的算法:新结果和局限性

Algorithms that Approximate Data Removal: New Results and Limitations

MIT(麻省理工学院)

Vinith M. Suriyakumar, Ashia C. Wilson

摘要:

我们研究了从使用经验风险最小化训练的机器学习模型中删除用户数据的问题。我们的重点是学习算法返回的经验风险最小化和近似unlearning算法,符合删除请求来流minibatches。利用无穷小的刀,我们开发了一个在线学习算法,是计算和内存效率。与现有的记忆高效的unlearning算法不同,我们的目标模型,最小化目标与非光滑正则化,如常用的$\ell_1$,弹性网,或核范数处罚。我们还提供泛化,删除能力和遗忘保证,与最先进的方法相一致。在各种基准数据集上,我们的算法根据经验改进了先前方法的运行时间,同时保持了相同的内存需求和测试精度。最后,我们打开了一个新的方向的调查证明,迄今为止介绍的所有近似unlearning算法无法unlearn在常见的超参数调整方法,如交叉验证,已被用来选择模型的问题设置。

英文摘要:

We study the problem of deleting user data from machine learning models trained using empirical risk minimization. Our focus is on learning algorithms which return the empirical risk minimizer and approximate unlearning algorithms that comply with deletion requests that come streaming minibatches. Leveraging the infintesimal jacknife, we develop an online unlearning algorithm that is both computationally and memory efficient. Unlike prior memory efficient unlearning algorithms, we target models that minimize objectives with non-smooth regularizers, such as the commonly used $\ell_1$, elastic net, or nuclear norm penalties. We also provide generalization, deletion capacity, and unlearning guarantees that are consistent with state of the art methods. Across a variety of benchmark datasets, our algorithm empirically improves upon the runtime of prior methods while maintaining the same memory requirements and test accuracy. Finally, we open a new direction of inquiry by proving that all approximate unlearning algorithms introduced so far fail to unlearn in problem settings where common hyperparameter tuning methods, such as cross-validation, have been used to select models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑