In the field of industrial inspection, Multimodal Large Language Models (MLLMs) have a high potential to renew the paradigms in practical applications due to their robust language capabilities and generalization abilities. However, despite their impressive problem-solving skills in many domains, MLLMs' ability in industrial anomaly detection has not been systematically studied. To bridge this gap, we present MMAD, the first-ever full-spectrum MLLMs benchmark in industrial Anomaly Detection. We defined seven key subtasks of MLLMs in industrial inspection and designed a novel pipeline to generate the MMAD dataset with 39,672 questions for 8,366 industrial images. With MMAD, we have conducted a comprehensive, quantitative evaluation of various state-of-the-art MLLMs. The commercial models performed the best, with the average accuracy of GPT-4o models reaching 74.9%. However, this result falls far short of industrial requirements. Our analysis reveals that current MLLMs still have significant room for improvement in answering questions related to industrial anomalies and defects. We further explore two training-free performance enhancement strategies to help models improve in industrial scenarios, highlighting their promising potential for future research.
AD-Copilot: A Vision-Language Assistant for Industrial Anomaly Detection via Visual In-context Comparison
Department of Computer Science and Engineering, Southern University of Science and Technology(计算机科学与工程系,南方科技大学);Macau University of Science and Technology(澳门科技大学);Tencent YouTu Lab(腾讯YouTu实验室);Nanjing University(南京大学);Shanghai Jiao Tong University(上海交通大学);Centre for Frontier AI Research, A*STAR(前沿人工智能研究中心,A*STAR)
School of Computer Science and Engineering, Beihang University(北京航空航天大学计算机科学与工程学院);School of Software, Beihang University(北京航空航天大学软件学院);Shandong Inspur Intelligent Production Technology Co., Ltd(山东浪潮智能生产技术有限公司)
FractalAD: A simple industrial anomaly detection method using fractal anomaly generation and backbone knowledge distillation
Shenzhen Institute of Artificial Intelligence and Robotics for Society(深圳市人工智能与机器人研究院);Nanjing University of Aeronautics and Astronautics(南京航空航天大学);Shanghai Jiaotong University(上海交通大学)
本文提出 FractalAD 工业异常检测方法,通过分形异常生成采样完整异常形态,并利用骨干知识蒸馏提取正常样本先验,以端到端语义分割网络实现高效训练与推理,在 MVTec AD 和 MVTec 3D-AD 数据集上取得有竞争力的结果。
Robust Distribution Alignment for Industrial Anomaly Detection under Distribution Shift
Institute for Infocomm Research, A*STAR(新加坡科研局信息通信研究院);Nanyang Technological University(南洋理工大学);South China University of Technology(华南理工大学);National University of Singapore(新加坡国立大学)
Noise Fusion-based Distillation Learning for Anomaly Detection in Complex Industrial Environments
College of Intelligent Robotics and Advanced Manufacturing, Fudan University(智能机器人与先进制造学院,复旦大学);Shanghai Jiao Tong University(上海交通大学);Shanghai AI Laboratory(上海人工智能实验室)
Multi-AD: Cross-Domain Unsupervised Anomaly Detection for Medical and Industrial Applications
Institute of Science Tokyo(东京科学研究所);Institute of Integrated Research(综合研究院);BioMedical Artificial Intelligence (BMAI) Research Unit(生物医学人工智能(BMAI)研究单元)
Beyond Geometry: Benchmarking and Consistency Reasoning for 3D Logical Anomaly Detection
School of Artificial Intelligence and Robotics, Hunan University(湖南大学人工智能与机器人学院);National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University(湖南大学机器人视觉感知与控制技术国家工程研究中心)
ADPretrain: Advancing Industrial Anomaly Detection via Anomaly Representation Pretraining
School of Information Science and Electronic Engineering, Shanghai Jiao Tong University(上海交通大学信息科学与电子工程学院);MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University(上海交通大学人工智能MOE重点实验室);College of Artificial Intelligence, Nanjing Agricultural University(南京农业大学人工智能学院);Key Laboratory of Livestock Farming Equipment, Ministry of Agriculture and Rural Affairs, Nanjing Agricultural University(南京农业大学农业机械化关键实验室)
Towards Efficient Pixel Labeling for Industrial Anomaly Detection and Localization
Jiangxi Normal University(江西师范大学);Southern University of Science and Technology(南方科技大学);Swansea University(斯旺西大学);Zhejiang University(浙江大学);Anhui University(安徽大学);Northwestern Polytechnical University(西北工业大学)
Progressive Boundary Guided Anomaly Synthesis for Industrial Anomaly Detection
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
Dual-Branch Reconstruction Network for Industrial Anomaly Detection with RGB-D Data
Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence, Jiangnan University(江南大学江苏省模式识别与计算智能工程实验室);College of Internet of Things Engineering, Jiangnan University(江南大学物联网工程学院)
Cross-Modal Mapping and Dual-Branch Reconstruction for 2D-3D Multimodal Industrial Anomaly Detection
CNR-ISASI, Institute of Applied Sciences and Intelligent Systems / National Research Council of Italy(CNR-ISASI应用科学与智能系统研究所/意大利国家研究理事会);CNR-STIIMA, Institute of Intelligent Industrial Technologies and Systems for Advanced Manufacturing(CNR-STIIMA智能工业技术与系统先进制造研究所);Institute d'Electronique de Microelectronique et de Nanotechnologie (IEMN) / Universite Polytechnique Hauts de France(微电子与纳米技术研究所(IEMN)/法国高等法国理工学院)
Anomaly Multi-classification in Industrial Scenarios: Transferring Few-shot Learning to a New Task
National Key Laboratory of Human-Machine Hybrid Augmented Intelligence(人机混合增强智能全国重点实验室);National Engineering Research Center for Visual Information and Applications(视觉信息与应用国家工程研究中心);Institute of Artificial Intelligence and Robotics(人工智能与机器人研究所);Xi’an Jiaotong University(西安交通大学);School of Informatics(信息学院);Xiamen University(厦门大学);College of Mechatronics and Control Engineering(机电与控制工程学院);Shenzhen University(深圳大学)
本文提出工业异常多分类新任务,通过结合 RelationNet 与 PatchCore、伪类数据生成及对比学习,在 MvTec AD 和 MvTec3D AD 上取得优于直接微调 ResNet 的性能。
Plan First, Judge Later, Run Better: A DMAIC-Inspired Agentic System for Industrial Anomaly Detection
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学);Shanghai University of Finance and Economics(上海财经大学);Technische Universität München(慕尼黑技术大学);Southwestern University of Finance and Economics(西南财经大学)
Evaluating Anomaly Detectors for Simulated Highly Imbalanced Industrial Classification Problems
Department of Computing and Software, McMaster University(计算与软件系,麦斯特大学);Center for Mechatronics and Hybrid Technologies, McMaster University(机电与混合技术中心,麦斯特大学)
ShaTS: A Shapley-based Explainability Method for Time Series Artificial Intelligence Models applied to Anomaly Detection in Industrial Internet of Things
Departamento de Ingeniería y Tecnología de Computadores, University of Murcia(计算机工程与技术部门,穆尔西亚大学)