arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2102.12567cs.LG
收藏

用于深度神经网络的高效分布外检测的草图曲率

Sketching Curvature for Efficient Out-of-Distribution Detection for Deep Neural Networks

Stanford University(斯坦福大学);Massachusetts Institute of Technology(麻省理工学院)

Apoorva Sharma, Navid Azizan, Marco Pavone

摘要:

为了在实时决策系统的感知管道内安全地部署深度神经网络(DNN),需要能够有效且准确地检测训练分布外(OoD)输入的保障措施。基于最近利用DNN的局部曲率来推理认知不确定性的工作,我们提出了OoD检测的草图曲率(SCOD),这是一个与架构无关的框架,用于为任何经过训练的DNN提供任务相关的认知不确定性估计。离线时,给定一个训练模型及其训练数据,SCOD使用矩阵草图工具来可追踪地计算Fisher信息矩阵的低秩近似,该矩阵表征权重空间中的哪些方向对训练数据的预测最有影响。在线上,我们通过测量与这些方向正交的扰动在新的测试输入下可以改变多少预测来估计不确定性。我们将SCOD应用于不同架构的预训练网络的几个任务,从回归到分类。我们证明,SCOD实现了相当或更好的OoD检测性能,相对于现有的基线具有较低的计算负担。

英文摘要:

In order to safely deploy Deep Neural Networks (DNNs) within the perception pipelines of real-time decision making systems, there is a need for safeguards that can detect out-of-training-distribution (OoD) inputs both efficiently and accurately. Building on recent work leveraging the local curvature of DNNs to reason about epistemic uncertainty, we propose Sketching Curvature of OoD Detection (SCOD), an architecture-agnostic framework for equipping any trained DNN with a task-relevant epistemic uncertainty estimate. Offline, given a trained model and its training data, SCOD employs tools from matrix sketching to tractably compute a low-rank approximation of the Fisher information matrix, which characterizes which directions in the weight space are most influential on the predictions over the training data. Online, we estimate uncertainty by measuring how much perturbations orthogonal to these directions can alter predictions at a new test input. We apply SCOD to pre-trained networks of varying architectures on several tasks, ranging from regression to classification. We demonstrate that SCOD achieves comparable or better OoD detection performance with lower computational burden relative to existing baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑