发表机构
INRIA; Université de la Polynésie française; University of Sheffield; Princeton University（法国国家信息与自动化研究所; 法属波利尼西亚大学; 谢菲尔德大学; 普林斯顿大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出一种基于ERM-RER变分公式的精确机器遗忘方法，通过最大化待遗忘数据上的经验风险并施加相对熵正则化，得到新的Gibbs算法，其分布与重训练一致，并由此构建了数据点重加权框架。
AI 中文摘要
本文提出了一种基于经验风险最小化与相对熵正则化（ERM-RER）的变分公式，用于实现Gibbs监督学习算法的精确遗忘。该方法在对待遗忘数据集上最大化期望经验风险，同时以相对于原始算法的相对熵进行正则化。优化变量是模型空间上的概率测度，其解为另一个Gibbs概率测度，代表一个新的Gibbs监督学习算法。该方法保证了精确遗忘，即新Gibbs算法在分布上与在保留数据集上从头重新训练所得的算法一致。作为副产品，本文获得了一个在ERM-RER中通过策略性选择参考测度和正则化因子来重新加权数据点的框架。在该框架中，精确遗忘是待遗忘数据点贡献被赋予零权重的特例。更一般地，根据特定参数的选择，数据点可以在ERM-RER问题中被上加权或下加权，以用于特定目的，例如控制Gibbs算法的泛化误差。这为在ERM-RER中对经典数据点重新加权开辟了新的建设性或对抗性视角。
英文摘要
In this paper, a method for achieving exact unlearning for Gibbs supervised learning algorithms is proposed using a variational formulation inspired by empirical risk minimization subject to relative entropy regularization (ERM-RER). Such a method consists of maximizing the expected empirical risk over the dataset to be unlearned subject to a regularization by relative entropy with respect to the original algorithm. The optimization variable is a probability measure on the models; and the solution is another Gibbs probability measure that represents a new Gibbs supervised learning algorithm. The method guarantees exact unlearning in the sense that the new Gibbs algorithm coincides in distribution with the algorithm that would have been obtained by retraining from scratch on the dataset to be retained. As a byproduct, a framework for reweighting data points in ERM-RER by strategically choosing both the reference measure and the regularization factor is obtained. In this framework, exact unlearning is the special case in which zero-weight is assigned to the contribution of the data points to be unlearned. More generally, depending on the choice of certain parameters, data points can be up-weighted or down-weighted in ERM-RER problems for particular purposes, e.g., controlling the generalization error of Gibbs algorithms. This paves the way for new constructive or adversarial views on classical reweighting data points in ERM-RER.
CommentsIn Proc. of the IEEE International Symposium on Information Theory (ISIT), Guangzhou, China, Jun., 2026. 2026 Jack Keil Wolf ISIT Student Paper Award
DOI：10.1109/ISIT62367.2026.11653818
