arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.15166cs.CVcs.AIcs.LG
收藏

通过深度感知移除遗忘特定方向实现类别反学习

Class Unlearning via Depth-Aware Removal of Forget-Specific Directions

Johns Hopkins University(约翰霍普金斯大学)

Arman Hatami, Romina Aalishah, Ilya E. Monosov

AI总结:

本文提出DAMP方法,通过深度感知移除遗忘特定方向,改进类别反学习的选性遗忘,同时更好地保留保留类性能并减少深层残留遗忘结构。

摘要:

机器反学习旨在在不重新训练模型的情况下移除目标知识。然而,在类别反学习中,降低遗忘类的准确性并不一定意味着真正的遗忘:遗忘的信息可能仍编码在内部表示中,而显着的遗忘可能源于分类器头部抑制而非表示移除。我们显示现有类别反学习方法往往表现出弱或负的选择性,保留遗忘类结构在深度表示中,或严重依赖最终层偏移。我们随后引入DAMP(通过投影的深度感知调节),一种单次、闭合形式的权重手术方法,可以在不使用梯度优化的情况下从预训练网络中移除遗忘特定方向。在每个阶段,DAMP在下一个可学习操作的输入空间中计算类别原型,提取遗忘方向作为相对于保留类原型的残差,并应用基于投影的更新以减少下游对这些方向的敏感性。为了保持实用性,DAMP使用从探测分离性导出的参数无关深度感知缩放规则,应用较小的编辑在早期层和较大的编辑在深层。该方法自然扩展到多类遗忘通过低秩子空间移除。在MNIST、CIFAR-10、CIFAR-100和Tiny ImageNet以及卷积和变换器架构上,DAMP比一些先前方法更接近再训练的黄金标准,改进了选择性遗忘的同时更好地保留保留类性能并减少深层残留遗忘结构。

英文摘要:

Machine unlearning aims to remove targeted knowledge from a trained model without the cost of retraining from scratch. In class unlearning, however, reducing accuracy on forget classes does not necessarily imply true forgetting: forgotten information can remain encoded in internal representations, and apparent forgetting may arise from classifier-head suppression rather than representational removal. We show that existing class-unlearning methods often exhibit weak or negative selectivity, preserve forget-class structure in deep representations, or rely heavily on final-layer bias shifts. We then introduce DAMP (Depth-Aware Modulation by Projection), a one-shot, closed-form weight-surgery method that removes forget-specific directions from a pretrained network without gradient-based optimization. At each stage, DAMP computes class prototypes in the input space of the next learnable operator, extracts forget directions as residuals relative to retain-class prototypes, and applies a projection-based update to reduce downstream sensitivity to those directions. To preserve utility, DAMP uses a parameter-free depth-aware scaling rule derived from probe separability, applying smaller edits in early layers and larger edits in deeper layers. The method naturally extends to multi-class forgetting through low-rank subspace removal. Across MNIST, CIFAR-10, CIFAR-100, and Tiny ImageNet, and across convolutional and transformer architectures, DAMP more closely resembles the retraining gold standard than some of the prior methods, improving selective forgetting while better preserving retain-class performance and reducing residual forget-class structure in deep layers.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑