arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.09453cs.AIcs.CV
收藏

MMAD:面向工业异常检测的多模态大语言模型综合基准

MMAD: A Comprehensive Benchmark for Multimodal Large Language Models in Industrial Anomaly Detection

Southern University of Science and Technology(南方科技大学);Tencent YouTu Lab(腾讯优图实验室);University of Alberta(阿尔伯塔大学);Shanghai Jiao Tong University(上海交通大学)

Xi Jiang, Jian Li, Hanqiu Deng, Yong Liu, Bin-Bin Gao, Yifeng Zhou, Jialin Li, Chengjie Wang, Feng Zheng

AI总结:

针对工业异常检测领域MLLMs能力缺乏系统评估的问题,本文提出首个全谱系基准MMAD,构建含3.9万余问题的数据集并评测前沿模型,发现商用模型表现最优但仍未达工业要求,还探索了两种免训练增强策略。

摘要:

在工业检测领域,多模态大语言模型(MLLMs)凭借其强大的语言能力和泛化能力,在实际应用中具有革新范式的巨大潜力。然而,尽管MLLMs在诸多领域展现出出色的问题解决能力,其在工业异常检测中的能力尚未得到系统研究。为填补这一空白,我们提出了MMAD——首个面向工业异常检测的全谱系MLLMs基准。我们定义了MLLMs在工业检测中的7项关键子任务,并设计了一套新颖的流水线生成MMAD数据集,该数据集涵盖8366张工业图像对应的39672个问题。借助MMAD,我们对多种前沿MLLMs开展了全面的定量评估。其中商用模型表现最佳,GPT-4o系列模型的平均准确率达到74.9%,但该结果远未达到工业应用要求。分析表明,当前MLLMs在回答工业异常与缺陷相关问题方面仍有显著提升空间。我们进一步探索了两种无需训练的性能增强策略,以帮助模型在工业场景中提升表现,凸显了其在未来研究中的可观潜力。

英文摘要:

In the field of industrial inspection, Multimodal Large Language Models (MLLMs) have a high potential to renew the paradigms in practical applications due to their robust language capabilities and generalization abilities. However, despite their impressive problem-solving skills in many domains, MLLMs' ability in industrial anomaly detection has not been systematically studied. To bridge this gap, we present MMAD, the first-ever full-spectrum MLLMs benchmark in industrial Anomaly Detection. We defined seven key subtasks of MLLMs in industrial inspection and designed a novel pipeline to generate the MMAD dataset with 39,672 questions for 8,366 industrial images. With MMAD, we have conducted a comprehensive, quantitative evaluation of various state-of-the-art MLLMs. The commercial models performed the best, with the average accuracy of GPT-4o models reaching 74.9%. However, this result falls far short of industrial requirements. Our analysis reveals that current MLLMs still have significant room for improvement in answering questions related to industrial anomalies and defects. We further explore two training-free performance enhancement strategies to help models improve in industrial scenarios, highlighting their promising potential for future research.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑