arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.11269cs.LG
收藏

超越类别子空间:用于单域模型中可靠分布外检测的教师引导训练

Beyond the Class Subspace: Teacher-Guided Training for Reliable Out-of-Distribution Detection in Single-Domain Models

Hong Yang, Devroop Kar, Qi Yu, Travis Desell, Alex Ororbia

AI总结:

本文提出教师引导训练方法,通过蒸馏多域教师模型的残差结构,提升单域模型在分布外检测中的可靠性,显著降低远OOD的误检率,同时保持在域性能。

摘要:

分布外(OOD)检测方法在多域基准上表现良好,但许多实际系统是在单域数据上训练的。我们展示这种模式会引发几何故障模式,即领域敏感崩溃(DSC):监督训练将特征压缩到低秩类别子空间并抑制携带领域转移信号的方向。我们提供理论证明,在DSC下,基于距离和logit的OOD分数对领域转移的敏感性丧失。然后我们引入教师引导训练(TGT),该方法在训练过程中将冻结多域教师(DINOv2)中的类别抑制残差结构蒸馏到学生模型中。教师和辅助头在训练后被丢弃,不增加推理开销。在八个单域基准上,TGT为基于距离的评分器带来了显著的远OOD FPR@95降低:MDS提高11.61个百分点,ViM提高10.78个百分点,kNN提高12.87个百分点(ResNet-50平均),同时保持或略微提高在域和分类准确性。

英文摘要:

Out-of-distribution (OOD) detection methods perform well on multi-domain benchmarks, yet many practical systems are trained on single-domain data. We show that this regime induces a geometric failure mode, Domain-Sensitivity Collapse (DSC): supervised training compresses features into a low-rank class subspace and suppresses directions that carry domain-shift signal. We provide theory showing that, under DSC, distance- and logit-based OOD scores lose sensitivity to domain shift. We then introduce Teacher-Guided Training (TGT), which distills class-suppressed residual structure from a frozen multi-domain teacher (DINOv2) into the student during training. The teacher and auxiliary head are discarded after training, adding no inference overhead. Across eight single-domain benchmarks, TGT yields large far-OOD FPR@95 reductions for distance-based scorers: MDS improves by 11.61 pp, ViM by 10.78 pp, and kNN by 12.87 pp (ResNet-50 average), while maintaining or slightly improving in-domain OOD and classification accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑