arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2111.09805cs.LG
收藏

DICE:利用稀疏化进行分布外检测

DICE: Leveraging Sparsification for Out-of-Distribution Detection

University of Wisconsin-Madison(威斯康星大学麦迪逊分校)

Yiyou Sun, Yixuan Li

摘要:

检测分发外(OOD)输入是在现实世界中安全部署机器学习模型的核心挑战。以前的方法通常依赖于从超参数化的权重空间导出的OOD分数,而在很大程度上忽略了稀疏化的作用。在本文中,我们揭示了重要的见解,依赖于不重要的权重和单位可以直接归因于OOD检测的脆性。为了缓解这个问题,我们提出了一个基于稀疏化的OOD检测框架,称为DICE。我们的核心思想是根据贡献的度量对权重进行排名,并选择性地使用最突出的权重来导出OOD检测的输出。我们提供了经验和理论的见解,表征和解释DICE提高OOD检测的机制。通过修剪噪声信号,DICE可证明降低了OOD数据的输出方差,从而使输出分布更清晰,与ID数据的分离性更强。我们证明了基于稀疏化的OOD检测在几个基准上的有效性,并建立了具有竞争力的性能。

英文摘要:

Detecting out-of-distribution (OOD) inputs is a central challenge for safely deploying machine learning models in the real world. Previous methods commonly rely on an OOD score derived from the overparameterized weight space, while largely overlooking the role of sparsification. In this paper, we reveal important insights that reliance on unimportant weights and units can directly attribute to the brittleness of OOD detection. To mitigate the issue, we propose a sparsification-based OOD detection framework termed DICE. Our key idea is to rank weights based on a measure of contribution, and selectively use the most salient weights to derive the output for OOD detection. We provide both empirical and theoretical insights, characterizing and explaining the mechanism by which DICE improves OOD detection. By pruning away noisy signals, DICE provably reduces the output variance for OOD data, resulting in a sharper output distribution and stronger separability from ID data. We demonstrate the effectiveness of sparsification-based OOD detection on several benchmarks and establish competitive performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑