VMAD：视觉增强的多模态大语言模型用于零样本异常检测
VMAD: Visual-enhanced Multimodal Large Language Model for Zero-Shot Anomaly Detection
AI总结：
VMAD通过结合视觉信息与多模态大语言模型，提升零样本异常检测的精度与分析能力，提出缺陷敏感结构学习和局部增强令牌压缩等方法，结合RIAD数据集验证了其有效性。
摘要：
零样本异常检测（ZSAD）通过建立文本提示与检查图像之间的特征映射，识别和定位以前未见过的对象，展现出在灵活工业制造中的研究价值。然而，现有ZSAD方法受限于封闭世界设置，难以处理预定义提示下的未知缺陷。最近，将多模态大语言模型（MLLMs）应用于工业异常检测（IAD）提供了一种可行的解决方案。与固定提示方法不同，MLLMs具有生成性范式，能够进行更适应的异常分析。然而，这种适应面临固有挑战，因为异常常出现在细粒度区域，并且与正常样本的视觉差异极小。为了解决这些挑战，我们提出了一种新的框架VMAD（视觉增强的MLLM异常检测），通过增强MLLM的视觉基于IAD知识和细粒度感知，同时提供精确的检测和全面的异常分析。具体来说，我们设计了一种缺陷敏感的结构学习方案，将视觉分支中的补丁相似性线索转移到我们的MLLM中以提高异常判别。此外，我们引入了一种新的视觉投影器，局部增强令牌压缩，该投影器挖掘多级特征以增强细粒度检测。此外，我们引入了现实工业异常检测（RIAD），一个包含详细异常描述和分析的综合性IAD数据集，为基于MLLM的IAD开发提供了宝贵的资源。在零样本基准测试中，包括MVTec-AD、Visa、WFDD和RIAD数据集的大量实验展示了我们的方法在状态-of-the-art方法上的优越性能。代码和数据集将很快发布。
英文摘要：
Zero-shot anomaly detection (ZSAD) recognizes and localizes anomalies in previously unseen objects by establishing feature mapping between textual prompts and inspection images, demonstrating excellent research value in flexible industrial manufacturing. However, existing ZSAD methods are limited by closed-world settings, struggling to unseen defects with predefined prompts. Recently, adapting Multimodal Large Language Models (MLLMs) for Industrial Anomaly Detection (IAD) presents a viable solution. Unlike fixed-prompt methods, MLLMs exhibit a generative paradigm with open-ended text interpretation, enabling more adaptive anomaly analysis. However, this adaption faces inherent challenges as anomalies often manifest in fine-grained regions and exhibit minimal visual discrepancies from normal samples. To address these challenges, we propose a novel framework VMAD (Visual-enhanced MLLM Anomaly Detection) that enhances MLLM with visual-based IAD knowledge and fine-grained perception, simultaneously providing precise detection and comprehensive analysis of anomalies. Specifically, we design a Defect-Sensitive Structure Learning scheme that transfers patch-similarities cues from visual branch to our MLLM for improved anomaly discrimination. Besides, we introduce a novel visual projector, Locality-enhanced Token Compression, which mines multi-level features in local contexts to enhance fine-grained detection. Furthermore, we introduce the Real Industrial Anomaly Detection (RIAD), a comprehensive IAD dataset with detailed anomaly descriptions and analyses, offering a valuable resource for MLLM-based IAD development. Extensive experiments on zero-shot benchmarks, including MVTec-AD, Visa, WFDD, and RIAD datasets, demonstrate our superior performance over state-of-the-art methods. The code and dataset will be available soon.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
