发表机构
Technische Universität Braunschweig（布伦瑞克工业大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
提出SAM3-O2D2零样本物体OOD检测方法，通过SAM3图像模型与目标检测器预测类别对比，实现高效检测，并在多个数据集上超越现有SOTA。
AI 中文摘要
目标检测器在医学影像、监控和自动驾驶等多个领域展现了卓越的性能。然而，在现实部署中遇到未见过的物体时，它们容易产生过度自信，从而引发潜在的安全问题。为解决这一问题，检测分布外（OOD）物体对于可靠的目标检测至关重要。现代方法利用如CLIP等基础模型的广泛语义知识进行事后少样本和零样本OOD检测。然而，这些方法通常在特征空间中进行OOD评估，这可能对目标检测器的定位误差和物体外观变化敏感。此外，当前最先进的（SOTA）零样本方法在推理时执行计算成本高昂的扩散过程。在本工作中，我们提出的零样本物体OOD检测方法SAM3-O2D2，以高效方式使用SAM3-图像基础模型。具体而言，我们仅使用目标检测器预测的类别来提示SAM3，并比较目标检测器与SAM3的预测。如果SAM3在相应位置也检测到物体，则该物体为分布内（ID）。如果SAM3未检测到提示的物体，则表明检测器的预测与图像内容不匹配，提示该物体为OOD。实验结果表明，我们的方法显著超越了迄今为止的零样本SOTA方法。具体而言，我们在两个ID数据集Pascal-VOC和BDD100K以及两个OOD数据集MS-COCO和OpenImages上均取得了新的SOTA AuROC和FPR95指标。
英文摘要
Object detectors have shown remarkable performance in various fields, among these medical imaging, surveillance, and autonomous driving. However, they are prone to overconfidence when encountering unseen objects in real-world deployments, causing potential safety issues. To address this, detecting out-of-distribution (OOD) objects is essential for reliable object detection. Modern approaches leverage the broad semantic knowledge of foundation models such as CLIP for post-hoc few- and zero-shot OOD detection. However, these methods typically perform OOD assessment in feature space, which can be sensitive to object detector localization errors and variations in object appearance. Moreover, the current state-of-the-art (SOTA) zero-shot method performs computationally costly diffusion in inference. In this work, for our proposed zero-shot object OOD detection method SAM3-O2D2, we employ the SAM3-image foundation model in an efficient manner. Specifically, we prompt SAM3 only with the object detector's predicted classes and compare the predictions of the object detector and SAM3. An object is in-distribution (ID), if SAM3 also detects an object at the corresponding location. If SAM3 does not detect the prompted object, this indicates a mismatch between the detector's prediction and the image content, suggesting that the object is OOD. Experimental results show that our method significantly surpasses the so-far zero-shot SOTA method. Specifically, we achieve new SOTA AuROC and FPR95 metrics over both ID datasets Pascal-VOC and BDD100K and both OOD datasets MS-COCO and OpenImages.
