面向流式遗忘的机器遗忘学习
Machine Unlearning for Streaming Forgetting
AI总结：
本文针对流式数据删除请求场景，提出一种将遗忘建模为分布偏移的流式遗忘范式及无需访问原始训练数据的高效算法，并给出非凸条件下的遗憾界理论保证。
摘要：
机器遗忘学习旨在从已训练好的模型中移除特定训练数据的知识。目前，机器遗忘学习方法通常以单一批次处理所有需要遗忘的数据，在收到请求时一次性移除相应知识。然而，在实际场景中，数据删除请求往往以流式方式出现，而非单一批次，这导致现有方法的效率和有效性下降。此类流式遗忘的挑战尚未得到广泛研究。本文针对流式遗忘请求带来的性能维持、效率和数据访问等挑战，提出了一种流式遗忘范式，将遗忘问题形式化为分布偏移问题。随后，我们估计改变后的分布，并提出一种新颖的流式遗忘算法，在无需访问原始训练数据的情况下实现高效的流式遗忘。理论分析证实了流式遗忘遗憾的$O(\sqrt{T} + V_T)$误差界，其中$V_T$表示在$T$个学习轮次中最优解的累计总变差。该理论保证在温和条件下即可成立，无需凸损失函数的强限制。在多种模型和数据集上的实验验证了我们所提方法的性能。
英文摘要：
Machine unlearning aims to remove knowledge of the specific training data in a well-trained model. Currently, machine unlearning methods typically handle all forgetting data in a single batch, removing the corresponding knowledge all at once upon request. However, in practical scenarios, requests for data removal often arise in a streaming manner rather than in a single batch, leading to reduced efficiency and effectiveness in existing methods. Such challenges of streaming forgetting have not been the focus of much research. In this paper, to address the challenges of performance maintenance, efficiency, and data access brought about by streaming unlearning requests, we introduce a streaming unlearning paradigm, formalizing the unlearning as a distribution shift problem. We then estimate the altered distribution and propose a novel streaming unlearning algorithm to achieve efficient streaming forgetting without requiring access to the original training data. Theoretical analyses confirm an $O(\sqrt{T} + V_T)$ error bound on the streaming unlearning regret, where $V_T$ represents the cumulative total variation in the optimal solution over $T$ learning rounds. This theoretical guarantee is achieved under mild conditions without the strong restriction of convex loss function. Experiments across various models and datasets validate the performance of our proposed method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
