arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.08083cs.CV
收藏

在自动驾驶中基于视觉基础模型的输入监控基准测试

Benchmarking Vision Foundation Models for Input Monitoring in Autonomous Driving

Continental AG(大陆汽车集团);Technical University of Munich(慕尼黑技术大学);University of Lübeck(吕贝克大学);University of Oxford(牛津大学);University of Wuppertal(伍珀塔尔大学)

Mert Keser, Halil Ibrahim Orhan, Niki Amini-Naieni, Gesina Schwalbe, Alois Knoll, Matthias Rottmann

AI总结:

本文提出了一种基于视觉基础模型和密度建模技术的统一方法,用于检测语义和协变量偏移,通过全面基准测试评估了VFM在复杂条件下的OOD分类能力,并证明其在识别高风险输入方面优于现有方法。

摘要:

深度神经网络(DNNs)在复杂开放世界领域如自动驾驶(AD)中仍面临分布偏移的挑战:对未知新对象(语义偏移)或风格如光照条件(协变量偏移)的鲁棒性无法保证。因此,可靠的操作时间监控器用于识别超出训练数据分布(OOD)场景至关重要。当前的OOD分类方法在复杂领域如AD中未经测试,检测偏移类型有限,甚至需要监督使用OOD样本。为应对不可预见的偏移,我们建立了一种围绕原则性、无监督和模型无关的方法框架,统一检测语义和协变量偏移:找到训练数据特征分布的完整模型,然后使用其在新点上的密度作为分布内(ID)分数。为实现此目的,我们提出将视觉基础模型(VFMs)作为特征提取器与密度建模技术结合。通过全面基准测试4种不同主干架构的VFMs和5种密度建模技术,我们提供了首个系统评估VFM在多样化条件下的OOD分类能力。与最先进的二元OOD分类方法的比较表明,VFM嵌入与密度估计在识别OOD输入方面优于现有方法。此外,我们证明了我们的方法能够检测可能在下游任务中导致错误的高风险输入,从而提高整体性能。总体而言,当VFM与稳健的密度建模技术结合时,有望在复杂视觉任务中实现模型无关、无监督、可靠的的安全监控。

英文摘要:

Deep neural networks (DNNs) remain challenged by distribution shifts in complex open-world domains like automated driving (AD): Robustness against yet unknown novel objects (semantic shift) or styles like lighting conditions (covariate shift) cannot be guaranteed. Hence, reliable operation-time monitors for identification of out-of-training-data-distribution (OOD) scenarios are imperative. Current approaches for OOD classification are untested for complex domains like AD, are limited in the kinds of shifts they detect, or even require supervision with OOD samples. To prepare for unanticipated shifts, we instead establish a framework around a principled, unsupervised and model-agnostic method that unifies detection of semantic and covariate shifts: Find a full model of the training data's feature distribution, to then use its density at new points as in-distribution (ID) score. To implement this, we propose to combine Vision Foundation Models (VFMs) as feature extractors with density modeling techniques. Through a comprehensive benchmark of 4 VFMs with different backbone architectures and 5 density-modeling techniques against established baselines, we provide the first systematic evaluation of OOD classification capabilities of VFMs across diverse conditions. A comparison with state-of-the-art binary OOD classification methods reveals that VFM embeddings with density estimation outperform existing approaches in identifying OOD inputs. Additionally, we show that our method detects high-risk inputs likely to cause errors in downstream tasks, thereby improving overall performance. Overall, VFMs, when coupled with robust density modeling techniques, are promising to realize model-agnostic, unsupervised, reliable safety monitors in complex vision tasks

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑