arXiv 2603.137792026-04-21cs.CVcs.AI

AD-Copilot：通过视觉上下文比较实现工业异常检测的视觉-语言助手

AD-Copilot: A Vision-Language Assistant for Industrial Anomaly Detection via Visual In-context Comparison

Department of Computer Science and Engineering, Southern University of Science and Technology（计算机科学与工程系，南方科技大学）；Macau University of Science and Technology（澳门科技大学）；Tencent YouTu Lab（腾讯YouTu实验室）；Nanjing University（南京大学）；Shanghai Jiao Tong University（上海交通大学）；Centre for Frontier AI Research, A*STAR（前沿人工智能研究中心，A*STAR）

Xi Jiang, Yue Guo, Jian Li, Yong Liu, Bin-Bin Gao, Hanqiu Deng, Jun Liu, Heng Zhao, Chengjie Wang, Feng Zheng

备注：: Code and models are released at https://github.com/jam-cc/AD-Copilot

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

AD-Copilot通过视觉上下文比较提升工业异常检测性能，采用新颖的数据筛选流程和比较编码器，在多个基准测试中取得优异成绩，超越人类专家水平。

摘要：

多模态大语言模型（MLLMs）在自然视觉理解上取得显著成功，但在工业异常检测（IAD）中表现欠佳，因为这些模型主要基于通用网络数据训练，与工业图像差异显著。此外，它们独立编码每张图像，只能在语言空间中比较图像，对关键的细微视觉差异不敏感。为解决这些问题，我们提出了AD-Copilot，一个通过视觉上下文比较专门用于IAD的交互式MLLM。我们首先设计了一个新颖的数据筛选流程，从稀疏标记的工业图像中挖掘检查知识，生成精确的样本用于描述、VQA和缺陷定位，从而获得富含语义信号的大型多模态数据集Chat-AD。在此基础上，AD-Copilot集成了一个新颖的比较编码器，通过配对图像特征之间的交叉注意力增强多图像细粒度感知，并采用多阶段策略结合领域知识逐步提升IAD技能。此外，我们引入了MMAD-BBox，一个带有边界框评估的扩展基准用于异常定位。实验表明，AD-Copilot在MMAD基准上达到82.3%的准确率，优于所有其他模型，无数据泄漏。在MMAD-BBox测试中，其表现比基线提高了3.35倍。AD-Copilot在其他专用和通用基准上也表现出色。值得注意的是，AD-Copilot在多个IAD任务中超越了人类专家水平，展示了其作为现实工业检查可靠助手的潜力。所有数据集和模型将发布给社区，以促进更广泛的利益。

英文摘要：

Multimodal Large Language Models (MLLMs) have achieved impressive success in natural visual understanding, yet they consistently underperform in industrial anomaly detection (IAD). This is because MLLMs trained mostly on general web data differ significantly from industrial images. Moreover, they encode each image independently and can only compare images in the language space, making them insensitive to subtle visual differences that are key to IAD. To tackle these issues, we present AD-Copilot, an interactive MLLM specialized for IAD via visual in-context comparison. We first design a novel data curation pipeline to mine inspection knowledge from sparsely labeled industrial images and generate precise samples for captioning, VQA, and defect localization, yielding a large-scale multimodal dataset Chat-AD rich in semantic signals for IAD. On this foundation, AD-Copilot incorporates a novel Comparison Encoder that employs cross-attention between paired image features to enhance multi-image fine-grained perception, and is trained with a multi-stage strategy that incorporates domain knowledge and gradually enhances IAD skills. In addition, we introduce MMAD-BBox, an extended benchmark for anomaly localization with bounding-box-based evaluation. The experiments show that AD-Copilot achieves 82.3% accuracy on the MMAD benchmark, outperforming all other models without any data leakage. In the MMAD-BBox test, it achieves a maximum improvement of $3.35\times$ over the baseline. AD-Copilot also exhibits excellent generalization of its performance gains across other specialized and general-purpose benchmarks. Remarkably, AD-Copilot surpasses human expert-level performance on several IAD tasks, demonstrating its potential as a reliable assistant for real-world industrial inspection. All datasets and models will be released for the broader benefit of the community.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。