一种面向任务无关持续学习-遗忘的统一梯度框架
A Unified Gradient-based Framework for Task-agnostic Continual Learning-Unlearning
AI总结：
本研究提出基于梯度的统一持续学习-遗忘框架UG-CLU，通过统一优化框架揭示CL与MU内在关联，引入保留流形约束等解决稳定性-可塑性困境，率先探索任务无关场景，实验验证其能协调增量学习、精准遗忘与知识稳定。
摘要：
深度模型的最新发展凸显了智能系统需结合用于知识获取的持续学习（CL）与用于数据移除的机器遗忘（MU），由此形成了持续学习-遗忘（CLU）范式。现有研究将CL和MU视为独立过程，而我们通过基于Kullback-Leibler散度最小化的统一优化框架揭示了二者的内在关联。该框架将近似CLU的梯度更新分解为四个组成部分：学习新知识、遗忘目标数据、保留已有知识，以及通过权重显著性进行调制。在连续的学习-遗忘周期中，平衡知识更新与保留是一项关键挑战。为解决这一稳定性-可塑性困境，我们引入了保留流形约束，为CLU迭代引入剩余海森矩阵补偿。我们设计了快慢权重自适应机制以高效近似二阶优化方向，结合自适应加权系数与平衡权重显著性掩码，提出了基于梯度的CLU统一实现框架。此外，我们率先提出了任务无关的CLU场景，除传统的任务感知设置外，还支持跨任务类别和随机样本级别的细粒度遗忘。实验表明，所提出的UG-CLU框架在多个数据集和模型架构上能有效协调增量学习、精准遗忘与知识稳定性，为动态合规的智能系统提供了理论基础与方法支撑。
英文摘要：
Recent advancements in deep models have highlighted the need for intelligent systems that combine continual learning (CL) for knowledge acquisition with machine unlearning (MU) for data removal, forming the Continual Learning-Unlearning (CLU) paradigm. While existing work treats CL and MU as separate processes, we reveal their intrinsic connection through a unified optimization framework based on Kullback-Leibler divergence minimization. This framework decomposes gradient updates for approximate CLU into four components: learning new knowledge, unlearning targeted data, preserving existing knowledge, and modulation via weight saliency. A critical challenge lies in balancing knowledge update and retention during sequential learning-unlearning cycles. To resolve this stability-plasticity dilemma, we introduce a remain-preserved manifold constraint to induce a remaining Hessian compensation for CLU iterations. A fast-slow weight adaptation mechanism is designed to efficiently approximate the second-order optimization direction, combined with adaptive weighting coefficients and a balanced weight saliency mask, proposing a unified implementation framework for gradient-based CLU. Furthermore, we pioneer task-agnostic CLU scenarios that support fine-grained unlearning at the cross-task category and random sample levels beyond the traditional task-aware setups. Experiments demonstrate that the proposed UG-CLU framework effectively coordinates incremental learning, precise unlearning, and knowledge stability across multiple datasets and model architectures, providing a theoretical foundation and methodological support for dynamic, compliant intelligent systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
