arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.08730cs.LGcs.CR
收藏

类别级机器去学习中的分类头偏差:诊断、缓解与评估

Classification-Head Bias in Class-Level Machine Unlearning: Diagnosis, Mitigation, and Evaluation

School of Computer Science and Cyber Engineering, Guangzhou University(计算机科学与电子工程学院,广州大学);School of Artificial Intelligence, Guangzhou University(人工智能学院,广州大学);Guangzhou Institute of Internet of Things(广州物联网研究院)

Weidong Zheng, Kongyang Chen, Yuanwei Guo, Yatie Xiao

AI总结:

本文揭示了类别级去学习中的偏差主导捷径,提出BiasShift和两种新的机制,通过引入新的指标量化偏差依赖性,实验表明方法在保持去学习性能的同时更稳定。

摘要:

类别级机器去学习旨在移除指定类别的影响,同时保持对保留类别的模型效用。现有方法通常通过保留集准确率、遗忘集准确率和去学习时间进行评估,但这些指标对内部遗忘机制的洞察有限。本文揭示了类别级去学习中的偏差主导捷径:通过减少最终分类头中的相应偏差项可以抑制遗忘类别的预测。我们首先分析了分类头偏差在softmax交叉熵训练下的梯度动态,解释了仅优化保留集为何倾向于减少缺失类别的偏差。基于此观察,我们引入BiasShift作为诊断基线,表明简单的偏差操作可以满足传统去学习指标,但会留下异常的偏差模式。为缓解过度的遗忘类偏差抑制,我们提出了两种偏差感知机制,即双阶段偏差梯度反转机制(TS-BGRM)和下界hinge正则化(LB-HR)。我们进一步引入了三个偏差导向的指标,包括偏差稳定性系数(BSC)、中位偏差间隙(MBG)和最小偏差分数(MBS),以量化偏差依赖性和潜在泄漏。在CIFAR-10、CIFAR-100和Tiny-ImageNet上的实验表明,所提出的方法在保持去学习性能的同时产生更稳定的偏差分布。我们已发布代码至{https://github.com/zwd2024/Beyond-the-Shadow-of-Bias-From-Classification-Head-Bias-to-Parameter-Redistribution}。

英文摘要:

Class-level machine unlearning aims to remove the influence of specified classes while preserving model utility on retained classes. Existing methods are commonly evaluated by retain-set accuracy, forget-set accuracy, and unlearning time, but these metrics provide limited insight into how forgetting is achieved internally. In this paper, we reveal a bias-dominated shortcut in class-level unlearning: the prediction of forgotten classes can be suppressed by decreasing the corresponding bias terms in the final classification head. We first analyze the gradient dynamics of classification-head biases under softmax cross-entropy training, explaining why retain-set-only optimization tends to reduce the biases of absent classes. Based on this observation, we introduce BiasShift as a diagnostic baseline, showing that simple bias manipulation can satisfy conventional unlearning metrics while leaving abnormal bias patterns that reveal forgotten labels. To mitigate excessive forgotten-class bias suppression, we propose two bias-aware mechanisms, namely Two-Stage Bias Gradient Reversal Mechanism (TS-BGRM) and Lower-Bound Hinge Regularization (LB-HR). We further introduce three bias-oriented metrics, including Bias Stability Coefficient (BSC), Median Bias Gap (MBG), and Minimal Bias Score (MBS), to quantify bias dependence and potential leakage. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet demonstrate that the proposed methods maintain competitive unlearning performance while producing more stable bias distributions. We have released our code at {https://github.com/zwd2024/Beyond-the-Shadow-of-Bias-From-Classification-Head-Bias-to-Parameter-Redistribution}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑