arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.07962cs.LG
收藏

你的算法是卸载还是再训练?

We Should Distinguish Unlearning From Untraining

Hasso Plattner Institute(哈索·普拉特纳研究所)

Eleni Triantafillou, Ahmed Imtiaz Humayun, Monica Ribero, Alexander Matt Turner, Michael C. Mozer, Georgios Kaissis

AI总结:

本文指出'unlearning'术语被滥用,区分'untraining'和'unlearning'两种不同问题,旨在澄清概念并推动研究进展。

摘要:

随着模型变大且训练数据增加,人们越来越关注如何在事后'删除'特定数据点或行为。这一目标被称为'machine unlearning'。本文认为'unlearning'术语被过度使用,不同研究努力涵盖两种不同的问题定义,但文献中未明确区分。这导致了模糊性、不恰当的度量标准和基线,以及结果解释困难。本文通过建立'untraining'和'unlearning'的基本区别来解决这一问题,其中'untraining'旨在逆转训练特定遗忘集的影响,而'unlearning'则旨在通过这些例子更广泛地去除其底层分布。我们讨论了这些问题的技术定义,并将文献中研究的问题设置映射到每个问题。我们希望启动澄清技术定义的讨论,并识别一组被忽视的研究问题,因为我们认为这是加速'unlearning'领域进展的关键缺失步骤。

英文摘要:

There has been a recent surge in interest about the question of how we can ``delete'' specific data points or behaviours from a trained model, a goal referred to as ``machine unlearning''. We argue that the umbrella term ``unlearning'' actually spans two distinct problem formulations, but the distinction between them has not yet been observed in literature. This causes ambiguity around when an unlearning algorithm is expected to work, leads to the use of inappropriate metrics and baselines when comparing algorithms to one another, difficulty in interpreting results, and missed opportunities for pursuing critical research directions. In this paper, we argue for the position that a fundamental distinction must be made between two notions that we refer to as unlearning and untraining. On one hand, untraining aims to reverse the effect of having trained on a given ``forget set'', i.e. to remove the influence that that specific set of examples had on the model during training. On the other hand, unlearning aims not just to remove the influence of those given examples, but also of the entire underlying subdistribution from which they were sampled (and thus e.g. the concept or model behaviour that those examples represent). We discuss technical definitions of these problems and map problem settings studied in the literature to each notion. By disambiguating technical definitions, our work aims to accelerate progress in this important field.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑