EMIT: 面向工业异常检测的多模态大语言模型增强方法：基于难度感知的GRPO
EMIT: Enhancing MLLMs for Industrial Anomaly Detection via Difficulty-Aware GRPO
AI总结：
针对多模态大语言模型在工业异常检测中效果受限的问题，提出EMIT框架，通过难度感知GRPO、多任务数据集构建及软提示等技术，显著提升模型检测性能。
摘要：
工业异常检测（IAD）对保障制造系统的安全性与可靠性至关重要。尽管多模态大语言模型（MLLMs）展现出强大的视觉-语言推理能力，但在缺乏领域适配的情况下，其在IAD任务中的效果仍较为有限。本研究提出EMIT——一个通过难度感知组相对策略优化（GRPO）增强MLLMs工业异常检测能力的统一框架。EMIT构建了多任务IAD数据集，并利用GPT生成的物体文本描述弥补缺陷图像的缺失。针对少样本异常检测场景，该框架整合了软提示与基于补丁级对比得到的热图引导对比嵌入。为更好地处理困难样本（即MLLM难以生成正确答案的样本），我们提出难度感知GRPO，在原始GRPO基础上引入响应重采样策略，确保采样响应中包含正确答案，同时设计优势重加权机制强化对困难数据样本的学习。在MMAD基准上的大量实验表明，EMIT显著提升了MLLMs的IAD性能，在七个任务上相较于基础模型InternVL3-8B平均提升7.77%。
英文摘要：
Industrial anomaly detection (IAD) plays a crucial role in maintaining the safety and reliability of manufacturing systems. While multimodal large language models (MLLMs) show strong vision-language reasoning abilities, their effectiveness in IAD remains limited without domain-specific adaptation. In this work, we propose EMIT, a unified framework that enhances MLLMs for IAD via difficulty-aware group relative policy optimization (GRPO). EMIT constructs a multi-task IAD dataset and utilizes GPT-generated object text descriptions to compensate for missing defective images. For few-shot anomaly detection, it integrates a soft prompt and heatmap-guided contrastive embeddings derived from patch-level comparisons. To better handle difficult data samples, i.e., cases where the MLLM struggles to generate correct answers, we propose a difficulty-aware GRPO that extends the original GRPO by incorporating a response resampling strategy to ensure the inclusion of correct answers in the sampled responses, as well as an advantage reweighting mechanism to strengthen learning from such difficult data samples. Extensive experiments on the MMAD benchmark demonstrate that EMIT significantly enhances the IAD performance of MLLMs, achieving an average improvement of 7.77\% over the base model (InternVL3-8B) across seven tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
