arXiv 2412.052932024-11-22cs.CVcs.LG

FodFoM：由基础模型生成的伪造离群数据打造性能更强的视觉分布外检测器

FodFoM: Fake Outlier Data by Foundation Models Creates Stronger Visual Out-of-Distribution Detector

Peng Cheng Laboratory（鹏城实验室）；Key Laboratory of Machine Intelligence and Advanced Computing（机器智能与先进计算重点实验室）；China United Network Communications Corporation Limited Guangdong Branch（中国联合网络通信有限公司广东省分公司）

Jiankang Chen, Ling Deng, Zhiyong Gan, Wei-Shi Zheng, Ruixuan Wang

备注：: 13 pages, 7 figures
发表信息：: Proceedings of the 32nd ACM International Conference on Multimedia, 2024

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对现有视觉OOD检测方法依赖高成本辅助数据或假设简化的问题，提出FodFoM框架，结合多类基础模型生成两种伪造离群图像用于训练，在多个基准上取得当前最优检测性能。

摘要：

分布外（OOD）检测是机器学习模型部署于开放世界应用时的关键技术，其核心挑战在于缓解模型对OOD数据的过度自信问题。近期采用辅助离群数据集或合成离群特征的方法虽已展现出可观的OOD检测性能，但受限于高昂的数据收集成本或过于简化的假设，存在明显局限。本文提出一种新型OOD检测框架FodFoM，创新性地结合多个基础模型，生成两类具有挑战性的伪造离群图像用于分类器训练。第一类伪造离群图像依托BLIP-2的图像字幕生成能力、CLIP的视觉-语言知识以及Stable Diffusion的图像生成能力，通过联合调用这些基础模型，构建出与分布内（ID）图像语义相似但存在本质差异的伪造离群图像。第二类则利用GroundingDINO的目标检测能力，通过模糊ID图像中的前景ID物体，构建纯背景图像作为离群样本。该框架可灵活适配多种现有OOD检测方法。大量实证评估表明，借助所构建的伪造图像训练的图像分类器，能够更精准地将真实OOD图像与ID图像区分开，在多个基准测试上取得了全新的SOTA（state-of-the-art，当前最优）OOD检测性能。代码已开源至https://github.com/Cverchen/ACMMM2024-FodFoM。

英文摘要：

Out-of-Distribution (OOD) detection is crucial when deploying machine learning models in open-world applications. The core challenge in OOD detection is mitigating the model's overconfidence on OOD data. While recent methods using auxiliary outlier datasets or synthesizing outlier features have shown promising OOD detection performance, they are limited due to costly data collection or simplified assumptions. In this paper, we propose a novel OOD detection framework FodFoM that innovatively combines multiple foundation models to generate two types of challenging fake outlier images for classifier training. The first type is based on BLIP-2's image captioning capability, CLIP's vision-language knowledge, and Stable Diffusion's image generation ability. Jointly utilizing these foundation models constructs fake outlier images which are semantically similar to but different from in-distribution (ID) images. For the second type, GroundingDINO's object detection ability is utilized to help construct pure background images by blurring foreground ID objects in ID images. The proposed framework can be flexibly combined with multiple existing OOD detection methods. Extensive empirical evaluations show that image classifiers with the help of constructed fake images can more accurately differentiate real OOD images from ID ones. New state-of-the-art OOD detection performance is achieved on multiple benchmarks. The code is available at \url{https://github.com/Cverchen/ACMMM2024-FodFoM}.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。