arXiv 2507.232572025-07-31cs.LGcs.AI

基于影响近似的高效机器遗忘

Efficient Machine Unlearning via Influence Approximation

School of Computer Science and Technology, University of Science and Technology of China（计算机科学与技术学院，中国科学技术大学）；School of Artificial Intelligence and Data Science, University of Science and Technology of China（人工智能与数据科学学院，中国科学技术大学）；Anhui Province Key Laboratory of Big Data Analysis and Application, School of Computer Science and Technology, University of Science and Technology of China（安徽省大数据分析与应用重点实验室，计算机科学与技术学院，中国科学技术大学）

Jiawei Liu, Chenwang Wu, Defu Lian, Enhong Chen

备注：: 12 pages, 4 figures

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对基于影响的机器遗忘方法计算开销大的问题，本文建立增量学习与机器遗忘的理论关联，提出IAU算法，在删除保证、效率和模型效用间实现更优平衡，性能优于现有方法。

摘要：

随着隐私问题日益凸显，旨在让机器学习模型“遗忘”特定训练数据的机器遗忘（machine unlearning）技术受到越来越多的关注。在现有方法中，基于影响的遗忘方法因无需重新训练即可估计单个训练样本对模型参数的影响，成为一种主流方法。然而，该方法需要针对所有训练样本和参数计算海森矩阵（Hessian matrix）及其逆矩阵，计算开销极高，难以应用于大规模模型和数据删除请求频繁的场景，凸显了遗忘的难度。受认知科学中“记忆比遗忘更容易”的结论启发，本文建立了记忆（增量学习）与遗忘（机器遗忘）之间的理论关联，使得可以从增量学习的角度解决机器遗忘问题。与遗忘过程中耗时的海森矩阵计算不同，增量学习（记忆）通常依赖更高效的梯度优化，这也印证了上述认知理论。基于这一关联，本文从增量视角提出了影响近似遗忘算法（Influence Approximation Unlearning, IAU），用于实现高效的机器遗忘。大量实验评估表明，IAU在删除保证、遗忘效率和模型效用之间实现了更优的平衡，且在多种数据集和模型架构上的表现均优于现有最优方法。本文代码已开源至https://github.com/Lolo1222/IAU。

英文摘要：

Due to growing privacy concerns, machine unlearning, which aims at enabling machine learning models to ``forget" specific training data, has received increasing attention. Among existing methods, influence-based unlearning has emerged as a prominent approach due to its ability to estimate the impact of individual training samples on model parameters without retraining. However, this approach suffers from prohibitive computational overhead arising from the necessity to compute the Hessian matrix and its inverse across all training samples and parameters, rendering it impractical for large-scale models and scenarios involving frequent data deletion requests. This highlights the difficulty of forgetting. Inspired by cognitive science, which suggests that memorizing is easier than forgetting, this paper establishes a theoretical link between memorizing (incremental learning) and forgetting (unlearning). This connection allows machine unlearning to be addressed from the perspective of incremental learning. Unlike the time-consuming Hessian computations in unlearning (forgetting), incremental learning (memorizing) typically relies on more efficient gradient optimization, which supports the aforementioned cognitive theory. Based on this connection, we introduce the Influence Approximation Unlearning (IAU) algorithm for efficient machine unlearning from the incremental perspective. Extensive empirical evaluations demonstrate that IAU achieves a superior balance among removal guarantee, unlearning efficiency, and comparable model utility, while outperforming state-of-the-art methods across diverse datasets and model architectures. Our code is available at https://github.com/Lolo1222/IAU.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。