arXiv 2406.022632024-06-04cs.CV

M3DM-NR：基于多模态去噪的RGB-3D抗噪工业异常检测

M3DM-NR: RGB-3D Noisy-Resistant Industrial Anomaly Detection via Multimodal Denoising

Shanghai Jiao Tong University（上海交通大学）；Youtu Lab（腾讯优图实验室）；Tencent（腾讯）

Chengjie Wang, Haokun Zhu, Jinlong Peng, Yue Wang, Ran Yi, Yunsheng Wu, Lizhuang Ma, Jiangning Zhang

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对实际场景中数据集存在噪声的问题，本文提出抗噪框架M3DM-NR，利用CLIP的多模态判别能力，通过三阶段（疑似参考选择、多模态去噪、特征对齐与融合）实现RGB-3D工业异常检测与分割，优于现有方法。

摘要：

现有的工业异常检测方法主要集中于使用纯净RGB图像进行无监督学习。然而，RGB和3D数据对于异常检测都至关重要，且在实际场景中数据集很少是完全干净的。为了解决上述挑战，本文初步探讨了RGB-3D多模态噪声异常检测，提出了一种新颖的抗噪M3DM-NR框架，以利用CLIP强大的多模态判别能力。M3DM-NR包含三个阶段：阶段I引入了疑似参考样本选择模块，利用初始特征提取提取的多模态特征从训练数据集中筛选少量正常样本，并通过疑似异常图计算模块生成疑似异常图，以聚焦异常区域作为参考。阶段II以参考样本的疑似异常图为参考，输入图像、点云和文本信息，通过模态内比较和多尺度聚合操作实现训练样本的去噪。最后，阶段III提出了点特征对齐、无监督特征融合、噪声判别核心集选择和决策层融合模块，以学习训练数据集的模式，从而在滤除噪声的同时实现异常检测和分割。大量实验表明，M3DM-NR在3D-RGB多模态噪声异常检测中优于最先进的方法。

英文摘要：

Existing industrial anomaly detection methods primarily concentrate on unsupervised learning with pristine RGB images. Yet, both RGB and 3D data are crucial for anomaly detection, and the datasets are seldom completely clean in practical scenarios. To address above challenges, this paper initially delves into the RGB-3D multi-modal noisy anomaly detection, proposing a novel noise-resistant M3DM-NR framework to leveraging strong multi-modal discriminative capabilities of CLIP. M3DM-NR consists of three stages: Stage-I introduces the Suspected References Selection module to filter a few normal samples from the training dataset, using the multimodal features extracted by the Initial Feature Extraction, and a Suspected Anomaly Map Computation module to generate a suspected anomaly map to focus on abnormal regions as reference. Stage-II uses the suspected anomaly maps of the reference samples as reference, and inputs image, point cloud, and text information to achieve denoising of the training samples through intra-modal comparison and multi-scale aggregation operations. Finally, Stage-III proposes the Point Feature Alignment, Unsupervised Feature Fusion, Noise Discriminative Coreset Selection, and Decision Layer Fusion modules to learn the pattern of the training dataset, enabling anomaly detection and segmentation while filtering out noise. Extensive experiments show that M3DM-NR outperforms state-of-the-art methods in 3D-RGB multi-modal noisy anomaly detection.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。