FG-OrIU: 通过特征-梯度正交性实现更好的增量遗忘
FG-OrIU: Towards Better Forgetting via Feature-Gradient Orthogonality for Incremental Unlearning
AI总结：
FG-OrIU通过特征和梯度正交约束实现深度不可逆的增量遗忘，解决现有方法中残留信息可恢复的问题。
摘要：
增量遗忘（IU）对于预训练模型响应连续数据删除请求至关重要，但现有方法主要抑制参数或混淆知识，缺乏对特征和梯度层面的显式约束，导致表面遗忘，残留信息仍可恢复。这种不彻底的遗忘可能引发安全漏洞并破坏保留平衡，尤其是在IU场景中。我们提出FG-OrIU（特征-梯度正交性用于增量遗忘），首个统一在特征和梯度层面施加正交约束的框架，以实现深度遗忘，遗忘效果不可逆。FG-OrIU通过奇异值分解（SVD）分解特征空间，将遗忘和保留类别的特征分离到不同的子空间中。然后施加双重约束：特征正交投影到遗忘和保留类别上，同时梯度正交投影防止遗忘知识的重新引入和对保留类别的干扰。此外，动态子空间适应将新遗忘的子空间与收缩的保留子空间合并，确保在连续遗忘任务中去除与保留之间的稳定平衡。大量实验验证了我们方法的有效性。
英文摘要：
Incremental unlearning (IU) is critical for pre-trained models to comply with sequential data deletion requests, yet existing methods primarily suppress parameters or confuse knowledge without explicit constraints on both feature and gradient level, resulting in \textit{superficial forgetting} where residual information remains recoverable. This incomplete forgetting risks security breaches and disrupts retention balance, especially in IU scenarios. We propose FG-OrIU (\textbf{F}eature-\textbf{G}radient \textbf{Or}thogonality for \textbf{I}ncremental \textbf{U}nlearning), the first framework unifying orthogonal constraints on both features and gradients level to achieve deep forgetting, where the forgetting effect is irreversible. FG-OrIU decomposes feature spaces via Singular Value Decomposition (SVD), separating forgetting and remaining class features into distinct subspaces. It then enforces dual constraints: feature orthogonal projection on both forgetting and remaining classes, while gradient orthogonal projection prevents the reintroduction of forgotten knowledge and disruption to remaining classes during updates. Additionally, dynamic subspace adaptation merges newly forgetting subspaces and contracts remaining subspaces, ensuring a stable balance between removal and retention across sequential unlearning tasks. Extensive experiments demonstrate the effectiveness of our method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
