arXiv 2507.192532025-07-25cs.CV

BridgeNet：一种用于桥接2D与3D工业异常检测的统一多模态框架

BridgeNet: A Unified Multimodal Framework for Bridging 2D and 3D Industrial Anomaly Detection

Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究院，中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）；Shenzhen University of Advanced Technology（深圳先进技术大学）；State Key Lab of IOTSC, Department of CIS, University of Macau（物联网科学国家重点实验室，澳门大学）

An Xiang, Zixuan Huang, Xitong Gao, Kejiang Ye, Cheng-zhong Xu

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出BridgeNet统一多模态框架，通过解耦RGB外观与点云可见深度、共享参数并结合两类异常生成器，桥接2D与3D工业异常检测，在MVTec-3D AD和Eyecandies上达到SOTA。

摘要：

面向2D对象的工业异常检测已受到广泛关注，并在异常检测（AD）方法方面取得进展。然而，仅利用2D信息识别3D深度异常是不充分的。尽管已有方法将深度信息显式融合到RGB图像中，或使用点云骨干网络提取深度特征，但由于不同模态信息之间存在差异，这两种方法在多模态场景中都难以充分表征3D信息。此外，由于工业数据中异常样本稀缺，尤其是在多模态场景下，有必要进行异常生成以模拟真实世界的异常样本。因此，我们提出一种新颖的统一多模态异常检测框架来解决这些问题。我们的贡献包括3个关键方面。（1）我们以简单方式从3D点云数据中提取可见深度信息，并使用2D RGB图像表征外观，从而解耦深度与外观，以支持统一的异常生成。（2）得益于灵活的输入表征，所提出的多尺度高斯异常生成器和统一纹理异常生成器能够在RGB和深度中生成更丰富的异常。（3）所有模块针对RGB和深度数据共享参数，有效桥接2D与3D异常检测。后续模块可直接利用来自两种模态的特征，而无需复杂融合。实验表明，我们的方法在MVTec-3D AD和Eyecandies数据集上优于最先进方法（SOTA）。代码可在https://github.com/Xantastic/BridgeNet获取。

英文摘要：

Industrial anomaly detection for 2D objects has gained significant attention and achieved progress in anomaly detection (AD) methods. However, identifying 3D depth anomalies using only 2D information is insufficient. Despite explicitly fusing depth information into RGB images or using point cloud backbone networks to extract depth features, both approaches struggle to adequately represent 3D information in multimodal scenarios due to the disparities among different modal information. Additionally, due to the scarcity of abnormal samples in industrial data, especially in multimodal scenarios, it is necessary to perform anomaly generation to simulate real-world abnormal samples. Therefore, we propose a novel unified multimodal anomaly detection framework to address these issues. Our contributions consist of 3 key aspects. (1) We extract visible depth information from 3D point cloud data simply and use 2D RGB images to represent appearance, which disentangles depth and appearance to support unified anomaly generation. (2) Benefiting from the flexible input representation, the proposed Multi-Scale Gaussian Anomaly Generator and Unified Texture Anomaly Generator can generate richer anomalies in RGB and depth. (3) All modules share parameters for both RGB and depth data, effectively bridging 2D and 3D anomaly detection. Subsequent modules can directly leverage features from both modalities without complex fusion. Experiments show our method outperforms state-of-the-art (SOTA) on MVTec-3D AD and Eyecandies datasets. Code available at: https://github.com/Xantastic/BridgeNet

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。