VLMDiff：利用视觉语言模型实现基于扩散的多类异常检测
VLMDiff: Leveraging Vision-Language Models for Multi-Class Anomaly Detection with Diffusion
AI总结：
针对多类图像异常检测的挑战，提出VLMDiff框架，集成视觉语言模型与潜在扩散模型，通过VLM提取正常图像描述作为扩散模型的条件，无需手动标注，提升异常检测性能并增强泛化性。
摘要：
在多样、多类的真实世界图像中检测视觉异常是一项重大挑战。我们提出VLMDiff，一种新颖的无监督多类视觉异常检测框架，将潜在扩散模型（LDM）与视觉语言模型（VLM）集成以增强异常定位与检测能力。具体而言，预训练VLM通过简单提示提取详细图像描述，作为LDM训练的额外条件。现有基于扩散的方法依赖合成噪声生成，限制泛化性且需逐类训练模型，阻碍可扩展性；而VLMDiff利用VLM获取正常图像的描述文本，无需手动标注。实验中，VLMDiff在Real-IAD数据集上将像素级PRO指标提升25个点，在COCO-AD数据集上提升8个点，优于现有扩散方法。
英文摘要：
Detecting visual anomalies in diverse, multi-class real-world images is a significant challenge. We introduce \ours, a novel unsupervised multi-class visual anomaly detection framework. It integrates a Latent Diffusion Model (LDM) with a Vision-Language Model (VLM) for enhanced anomaly localization and detection. Specifically, a pre-trained VLM with a simple prompt extracts detailed image descriptions, serving as additional conditioning for LDM training. Current diffusion-based methods rely on synthetic noise generation, limiting their generalization and requiring per-class model training, which hinders scalability. \ours, however, leverages VLMs to obtain normal captions without manual annotations or additional training. These descriptions condition the diffusion model, learning a robust normal image feature representation for multi-class anomaly detection. Our method achieves competitive performance, improving the pixel-level Per-Region-Overlap (PRO) metric by up to 25 points on the Real-IAD dataset and 8 points on the COCO-AD dataset, outperforming state-of-the-art diffusion-based approaches. Code is available at https://github.com/giddyyupp/VLMDiff.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
