AnomalyGPT：使用大型视觉-语言模型检测工业异常
AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models
AI总结：
提出AnomalyGPT，利用大型视觉-语言模型进行工业异常检测，无需手动阈值，支持少样本学习，在MVTec-AD上达到SOTA性能。
摘要：
大型视觉-语言模型（LVLMs），如MiniGPT-4和LLaVA，已展现出理解图像的能力，并在各种视觉任务中取得了显著性能。尽管由于广泛的训练数据集，它们在识别常见物体方面能力强大，但缺乏特定领域的知识，对物体内部局部细节的理解较弱，这阻碍了它们在工业异常检测（IAD）任务中的有效性。另一方面，大多数现有的IAD方法仅提供异常分数，需要手动设置阈值来区分正常和异常样本，这限制了它们的实际应用。在本文中，我们探索利用LVLM来解决IAD问题，并提出AnomalyGPT，一种基于LVLM的新型IAD方法。我们通过模拟异常图像并为每张图像生成相应的文本描述来生成训练数据。我们还采用图像解码器提供细粒度的语义，并设计了一个提示学习器，使用提示嵌入对LVLM进行微调。我们的AnomalyGPT消除了手动阈值调整的需要，从而直接评估异常的存在和位置。此外，AnomalyGPT支持多轮对话，并展现出令人印象深刻的少样本上下文学习能力。仅需一个正常样本，AnomalyGPT在MVTec-AD数据集上达到了最先进的性能，准确率为86.1%，图像级AUC为94.1%，像素级AUC为95.3%。代码可在https://github.com/CASIA-IVA-Lab/AnomalyGPT获取。
英文摘要：
Large Vision-Language Models (LVLMs) such as MiniGPT-4 and LLaVA have demonstrated the capability of understanding images and achieved remarkable performance in various visual tasks. Despite their strong abilities in recognizing common objects due to extensive training datasets, they lack specific domain knowledge and have a weaker understanding of localized details within objects, which hinders their effectiveness in the Industrial Anomaly Detection (IAD) task. On the other hand, most existing IAD methods only provide anomaly scores and necessitate the manual setting of thresholds to distinguish between normal and abnormal samples, which restricts their practical implementation. In this paper, we explore the utilization of LVLM to address the IAD problem and propose AnomalyGPT, a novel IAD approach based on LVLM. We generate training data by simulating anomalous images and producing corresponding textual descriptions for each image. We also employ an image decoder to provide fine-grained semantic and design a prompt learner to fine-tune the LVLM using prompt embeddings. Our AnomalyGPT eliminates the need for manual threshold adjustments, thus directly assesses the presence and locations of anomalies. Additionally, AnomalyGPT supports multi-turn dialogues and exhibits impressive few-shot in-context learning capabilities. With only one normal shot, AnomalyGPT achieves the state-of-the-art performance with an accuracy of 86.1%, an image-level AUC of 94.1%, and a pixel-level AUC of 95.3% on the MVTec-AD dataset. Code is available at https://github.com/CASIA-IVA-Lab/AnomalyGPT.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
