LINe:利用重要神经元的分布外检测
LINe: Out-of-Distribution Detection by Leveraging Important Neurons
摘要:
量化输入样本的不确定性非常重要,特别是在自动驾驶和医疗保健等关键任务领域,这些领域对分布外(OOD)数据的故障预测可能会导致大问题。OOD检测问题从根本上开始于模型无法表达它所不知道的东西。事后OOD检测方法被广泛探索,因为它们不需要额外的重新训练过程,这可能会降低模型的性能并增加训练成本。在这项研究中,从神经元在模型的深层代表高级功能的角度来看,我们引入了一个新的方面来分析模型输出之间的差异分布数据和OOD数据。我们提出了一种新的方法,利用重要神经元(LINe),事后分布外检测。基于Shapley值的修剪通过仅选择用于预测特定类别的输入数据的高贡献神经元并掩蔽其余神经元来减少噪声输出的影响。激活裁剪将高于某个阈值的所有值固定为相同的值,允许LINe平等地对待所有类特定的特征,并且只考虑分布和OOD数据之间激活特征数量差异的差异。在CIFAR-10、CIFAR-100和ImageNet数据集上的综合实验验证了所提出方法的有效性,其性能优于最先进的事后OOD检测方法。
英文摘要:
It is important to quantify the uncertainty of input samples, especially in mission-critical domains such as autonomous driving and healthcare, where failure predictions on out-of-distribution (OOD) data are likely to cause big problems. OOD detection problem fundamentally begins in that the model cannot express what it is not aware of. Post-hoc OOD detection approaches are widely explored because they do not require an additional re-training process which might degrade the model's performance and increase the training cost. In this study, from the perspective of neurons in the deep layer of the model representing high-level features, we introduce a new aspect for analyzing the difference in model outputs between in-distribution data and OOD data. We propose a novel method, Leveraging Important Neurons (LINe), for post-hoc Out of distribution detection. Shapley value-based pruning reduces the effects of noisy outputs by selecting only high-contribution neurons for predicting specific classes of input data and masking the rest. Activation clipping fixes all values above a certain threshold into the same value, allowing LINe to treat all the class-specific features equally and just consider the difference between the number of activated feature differences between in-distribution and OOD data. Comprehensive experiments verify the effectiveness of the proposed method by outperforming state-of-the-art post-hoc OOD detection methods on CIFAR-10, CIFAR-100, and ImageNet datasets.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。