arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.17816cs.CVcs.LG
收藏

基于神经坍缩的特征分离用于分布外检测

Pursuing Feature Separation based on Neural Collapse for Out-of-Distribution Detection

Cranberry-Lemon University(蔓越莓柠檬大学);University of the Witwatersrand(金山大学)

Yingwen Wu, Ruiji Yu, Xinwen Cheng, Zhengbao He, Xiaolin Huang

AI总结:

本文提出基于神经坍缩的特征分离损失,将OOD特征约束在ID特征正交子空间,在多个基准上实现SOTA的OOD检测性能。

摘要:

在开放世界中,检测分布外(OOD)数据(其标签与分布内(ID)样本的标签不相交)对于可靠的深度神经网络(DNNs)至关重要。为了获得更好的检测性能,一类方法提出使用辅助OOD数据集对模型进行微调,通过在模型输出上定义的分离损失来放大ID与OOD数据之间的差异。然而,这些研究均未考虑扩大特征差异,而特征差异相比输出差异应更为有效。主要困难在于OOD样本的多样性,这使得描述其特征分布变得困难,更不用说设计损失函数将其与ID特征分离了。本文基于ID特征的一种聚合特性——神经坍缩(NC),巧妙地规避了这一问题。NC意味着同一类内ID样本的倒数第二层特征与该类最后一层权重几乎相同。基于这一特性,我们提出了一种简单而有效的损失函数,称为分离损失(Separation Loss),它将OOD数据的特征约束在与由NC形成的ID特征主子空间正交的子空间中。通过这种方式,ID和OOD样本的特征通过不同的维度被分离。通过优化特征分离损失而非单纯扩大输出差异,我们的检测方法在CIFAR10、CIFAR100和ImageNet基准上取得了SOTA性能,且无需任何额外的数据增强或采样,这证明了特征分离在OOD检测中的重要性。代码可在https://github.com/Wuyingwen/Pursuing-Feature-Separation-for-OOD-Detection获取。

英文摘要:

In the open world, detecting out-of-distribution (OOD) data, whose labels are disjoint with those of in-distribution (ID) samples, is important for reliable deep neural networks (DNNs). To achieve better detection performance, one type of approach proposes to fine-tune the model with auxiliary OOD datasets to amplify the difference between ID and OOD data through a separation loss defined on model outputs. However, none of these studies consider enlarging the feature disparity, which should be more effective compared to outputs. The main difficulty lies in the diversity of OOD samples, which makes it hard to describe their feature distribution, let alone design losses to separate them from ID features. In this paper, we neatly fence off the problem based on an aggregation property of ID features named Neural Collapse (NC). NC means that the penultimate features of ID samples within a class are nearly identical to the last layer weight of the corresponding class. Based on this property, we propose a simple but effective loss called Separation Loss, which binds the features of OOD data in a subspace orthogonal to the principal subspace of ID features formed by NC. In this way, the features of ID and OOD samples are separated by different dimensions. By optimizing the feature separation loss rather than purely enlarging output differences, our detection achieves SOTA performance on CIFAR10, CIFAR100 and ImageNet benchmarks without any additional data augmentation or sampling, demonstrating the importance of feature separation in OOD detection. Code is available at https://github.com/Wuyingwen/Pursuing-Feature-Separation-for-OOD-Detection.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑