Myriad：面向工业异常检测的融合视觉专家的大多模态模型
Myriad: Large Multimodal Model by Applying Vision Experts for Industrial Anomaly Detection
AI总结：
针对传统工业异常检测方法场景适配性差、大多模态模型因模态鸿沟难以利用相关知识的问题，提出融合视觉专家的Myriad模型，通过异常图引导与特征调制实现更优的检测性能与指令交互能力。
摘要：
受训练配置限制，传统工业异常检测（IAD）方法需为每个部署场景训练专属模型，难以满足现代设计与制造的需求。与之相对，大多模态模型（LMMs）已在各类视觉任务上展现出卓越的泛化能力，其感知与理解能力预示着将LMMs应用于IAD任务的潜力。但我们观察到，尽管LMMs在文本域拥有丰富的工业异常检测相关知识，却因文本与视觉域之间的模态鸿沟无法有效利用这些知识。\n为激活LMMs中的相关知识并使其适配异常检测任务，我们引入现有IAD方法作为视觉专家，提出一种融合视觉专家的工业异常检测大多模态模型（缩写为Myriad）。具体而言，我们利用视觉专家生成的异常图作为LMMs的引导，指导视觉模型更多关注异常区域；随后通过适配器调制视觉特征以适配异常检测任务，将调制后的视觉特征与视觉专家引导、人类指令一同输入语言模型，生成最终输出。\n在MVTec-AD、VisA和PCB Bank基准上开展的大量实验表明，所提方法不仅性能优于现有最优方法，还在IAD领域继承了LMMs的灵活性与指令遵循能力。源代码与预训练模型已公开于https://github.com/tzjtatata/Myriad。
英文摘要：
Due to the training configuration, traditional industrial anomaly detection (IAD) methods have to train a specific model for each deployment scenario, which is insufficient to meet the requirements of modern design and manufacturing. On the contrary, large multimodal models~(LMMs) have shown eminent generalization ability on various vision tasks, and their perception and comprehension capabilities imply the potential of applying LMMs on IAD tasks. However, we observe that even though the LMMs have abundant knowledge about industrial anomaly detection in the textual domain, the LMMs are unable to leverage the knowledge due to the modality gap between textual and visual domains. To stimulate the relevant knowledge in LMMs and adapt the LMMs towards anomaly detection tasks, we introduce existing IAD methods as vision experts and present a novel large multimodal model applying vision experts for industrial anomaly detection~(abbreviated to {Myriad}). Specifically, we utilize the anomaly map generated by the vision experts as guidance for LMMs, such that the vision model is guided to pay more attention to anomalous regions. Then, the visual features are modulated via an adapter to fit the anomaly detection tasks, which are fed into the language model together with the vision expert guidance and human instructions to generate the final outputs. Extensive experiments are applied on MVTec-AD, VisA, and PCB Bank benchmarks demonstrate that our proposed method not only performs favorably against state-of-the-art methods, but also inherits the flexibility and instruction-following ability of LMMs in the field of IAD. Source code and pre-trained models are publicly available at \url{https://github.com/tzjtatata/Myriad}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
