arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2306.02879cs.LG
收藏

神经元激活覆盖度:重新思考分布外检测与泛化

Neuron Activation Coverage: Rethinking Out-of-distribution Detection and Generalization

City University of Hong Kong(香港城市大学);The University of Tokyo(东京大学)

Yibing Liu, Chris Xing Tian, Haoliang Li, Lei Ma, Shiqi Wang

AI总结:

该研究从神经元激活视角研究分布外(OOD)问题,提出神经元激活覆盖度(NAC)指标,既实现了更优的OOD检测效果,又可作为评估模型鲁棒性的有效准则。

摘要:

当神经网络遇到与训练数据分布(即分布内(InD))显著偏离的数据时,通常会出现分布外(OOD)问题。本文从神经元激活的角度研究OOD问题。我们首先通过同时考虑神经元输出及其对模型决策的影响,构建了神经元激活状态的公式化表达。随后,为刻画神经元与OOD问题之间的关联,我们提出了神经元激活覆盖度(NAC)——一种衡量InD数据下神经元行为的简单指标。借助NAC,我们发现:1)基于神经元行为可在很大程度上区分InD和OOD输入,这大幅简化了OOD检测问题,在三个基准数据集(CIFAR-10、CIFAR-100和ImageNet-1K)上的表现优于21种现有方法。2)NAC与模型泛化能力之间的正相关关系在不同架构和数据集上始终成立,这使得基于NAC的准则可用于评估模型鲁棒性。与主流的InD验证准则相比,我们表明NAC不仅能选出更鲁棒的模型,还与OOD测试性能具有更强的相关性。

英文摘要:

The out-of-distribution (OOD) problem generally arises when neural networks encounter data that significantly deviates from the training data distribution, i.e., in-distribution (InD). In this paper, we study the OOD problem from a neuron activation view. We first formulate neuron activation states by considering both the neuron output and its influence on model decisions. Then, to characterize the relationship between neurons and OOD issues, we introduce the \textit{neuron activation coverage} (NAC) -- a simple measure for neuron behaviors under InD data. Leveraging our NAC, we show that 1) InD and OOD inputs can be largely separated based on the neuron behavior, which significantly eases the OOD detection problem and beats the 21 previous methods over three benchmarks (CIFAR-10, CIFAR-100, and ImageNet-1K). 2) a positive correlation between NAC and model generalization ability consistently holds across architectures and datasets, which enables a NAC-based criterion for evaluating model robustness. Compared to prevalent InD validation criteria, we show that NAC not only can select more robust models, but also has a stronger correlation with OOD test performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑