遗忘-保留困境：持续学习中的认证遗忘理论
The Forgetting-Retention Dilemma: Certified Unlearning Theory in Continual Learning
AI总结：
本文建立持续学习与机器遗忘之间的理论桥梁，通过定义后遗忘超额风险揭示保留历史知识与定向遗忘的权衡，并分析梯度法与海森法在遗忘损失和存储开销上的优劣，提出混合策略。
摘要：
机器遗忘旨在消除训练模型中特定数据的影响以保护隐私。然而，在持续学习（CL）中，模型在动态数据集上顺序更新，这带来了重大挑战。当前认证遗忘算法的一个主要局限是未能考虑CL框架中固有的复杂累积模型演化。在这项工作中，我们建立了连接CL和机器遗忘的第一个理论基础。我们将CL的遗忘目标表述为最小化后遗忘超额风险，该风险分解为CL超额风险和遗忘损失，刻画了保留历史知识与定向遗忘之间的基本权衡。在温和假设下，我们首先为非凸模型中的CL超额风险建立了上界。然后，我们将两种认证遗忘方法（基于梯度和基于海森）适配到CL框架。我们的分析表明，虽然基于梯度的方法在最小化遗忘损失方面不如基于海森的方法有效，但它具有几乎零存储开销以实现遗忘的独特优势。这一见解启发了一种混合策略，该策略在降低存储成本的同时保持后遗忘性能。实验结果进一步验证了我们的理论发现。
英文摘要：
Machine unlearning aims to eliminate the influence of specific data from trained models to safeguard privacy. However, this presents a significant challenge in the context of continual learning (CL), where models update sequentially on dynamic datasets. A major limitation is that current certified unlearning algorithms fail to account for the complex, cumulative model evolution inherent to CL framework. In this work, we establish the first theoretical foundation bridging CL and machine unlearning. We formulate the CL's unlearning objective as the minimization of post-unlearning excess risk, which decomposes into CL excess risk and unlearning loss, characterizing the fundamental trade-off between preserving historical knowledge and targeted forgetting. Under mild assumptions, we first establish an upper bound for the CL excess risk in non-convex models. We then adapt two certified unlearning approaches, gradient-based and Hessian-based, to the CL framework. Our analysis reveals that while the gradient-based approach is less effective than the Hessian-based method in minimizing unlearning loss, it offers the distinct advantage of nearly zero storage overhead for enabling unlearning. This insight motivates a hybrid strategy that reduces storage costs while maintaining post-unlearning performance. Experimental results further validate our theoretical findings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
