基于混合融合的多模态工业异常检测
Multimodal Industrial Anomaly Detection via Hybrid Fusion
AI总结：
针对多模态工业异常检测中特征直接拼接导致干扰的问题，提出采用混合融合方案的M3DM模型，通过对比学习、决策层融合及点特征对齐提升检测精度，在MVTec-3D AD数据集上优于SOTA方法。
摘要：
基于2D的工业异常检测已被广泛讨论，然而基于3D点云和RGB图像的多模态工业异常检测仍有许多未触及的领域。现有的多模态工业异常检测方法直接拼接多模态特征，这会导致特征间的强烈干扰并损害检测性能。在本文中，我们提出了Multi-3D-Memory (M3DM)，一种具有混合融合方案的新型多模态异常检测方法：首先，我们设计了一种基于块级对比学习的无监督特征融合，以促进不同模态特征之间的交互；其次，我们使用具有多个记忆库的决策层融合以避免信息丢失，并采用额外的新颖性分类器来做出最终决定。我们进一步提出了一种点特征对齐操作，以更好地对齐点云和RGB特征。大量实验表明，我们的多模态工业异常检测模型在MVTec-3D AD数据集上的检测和分割精度均优于现有最先进 (SOTA) 方法。代码可在 https://github.com/nomewang/M3DM 获取。
英文摘要：
2D-based Industrial Anomaly Detection has been widely discussed, however, multimodal industrial anomaly detection based on 3D point clouds and RGB images still has many untouched fields. Existing multimodal industrial anomaly detection methods directly concatenate the multimodal features, which leads to a strong disturbance between features and harms the detection performance. In this paper, we propose Multi-3D-Memory (M3DM), a novel multimodal anomaly detection method with hybrid fusion scheme: firstly, we design an unsupervised feature fusion with patch-wise contrastive learning to encourage the interaction of different modal features; secondly, we use a decision layer fusion with multiple memory banks to avoid loss of information and additional novelty classifiers to make the final decision. We further propose a point feature alignment operation to better align the point cloud and RGB features. Extensive experiments show that our multimodal industrial anomaly detection model outperforms the state-of-the-art (SOTA) methods on both detection and segmentation precision on MVTec-3D AD dataset. Code is available at https://github.com/nomewang/M3DM.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
