CUFG:由遗忘梯度引导的课程式遗忘
CUFG: Curriculum Unlearning Guided by the Forgetting Gradient
AI总结:
本文提出CUFG框架,通过遗忘梯度引导的梯度校正器和从易到难的课程式遗忘策略,提升机器遗忘的稳定性与可靠性,缩小与Retrain金标准的差距。
摘要:
随着隐私与安全成为人工智能的核心议题,机器遗忘——即从模型中擦除特定知识的能力——受到越来越多的关注。然而,现有方法过度强调效率和激进遗忘,带来了显著局限。特别是梯度上升、影响函数和随机标签噪声等激进干预手段可能破坏模型权重的稳定性,导致模型崩溃并降低可靠性。为解决这一问题,我们提出CUFG(通过遗忘梯度实现的课程式遗忘),这是一个新颖的框架,通过在遗忘机制和数据调度策略两方面的创新来增强近似遗忘的稳定性。具体而言,CUFG集成了由遗忘梯度引导的新型梯度校正器,用于基于微调的遗忘,并引入课程式遗忘范式,从易到难逐步遗忘。这些创新通过实现更稳定、渐进的遗忘,缩小了与金标准Retrain方法之间的差距,从而同时提升了有效性和可靠性。此外,我们认为课程式遗忘这一概念具有重要的研究潜力,并为机器遗忘领域的发展提供了前瞻性见解。在多种遗忘场景下进行的大量实验验证了我们方法及CUFG的合理性和有效性。代码可在https://anonymous.4open.science/r/CUFG-6375获取。
英文摘要:
As privacy and security take center stage in AI, machine unlearning, the ability to erase specific knowledge from models, has garnered increasing attention. However, existing methods overly prioritize efficiency and aggressive forgetting, which introduces notable limitations. In particular, radical interventions like gradient ascent, influence functions, and random label noise can destabilize model weights, leading to collapse and reduced reliability. To address this, we propose CUFG (Curriculum Unlearning via Forgetting Gradients), a novel framework that enhances the stability of approximate unlearning through innovations in both forgetting mechanisms and data scheduling strategies. Specifically, CUFG integrates a new gradient corrector guided by forgetting gradients for fine-tuning-based unlearning and a curriculum unlearning paradigm that progressively forgets from easy to hard. These innovations narrow the gap with the gold-standard Retrain method by enabling more stable and progressive unlearning, thereby improving both effectiveness and reliability. Furthermore, we believe that the concept of curriculum unlearning has substantial research potential and offers forward-looking insights for the development of the MU field. Extensive experiments across various forgetting scenarios validate the rationale and effectiveness of our approach and CUFG. Codes are available at https://anonymous.4open.science/r/CUFG-6375.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。