arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.05850cs.CV
收藏

Align3D-AD:跨模态特征对齐与双提示学习用于零样本3D异常检测

Align3D-AD: Cross-Modal Feature Alignment and Dual-Prompt Learning for Zero-shot 3D Anomaly Detection

Smart Manufacturing Thrust, The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州)智能制造方向);The Hong Kong University of Science and Technology(香港科技大学);College of Mechanical and Vehicle Engineering, Hunan University(湖南大学机械与车辆工程学院)

Letian Bai, Xuanming Cao, Juan Du, Chengyu Tao

AI总结:

本文提出Align3D-AD框架,通过跨模态特征对齐和双提示学习解决零样本3D异常检测中的领域差距问题,实验表明其在多个数据集上均优于现有方法。

摘要:

零样本3D异常检测旨在无需目标类别的训练数据即可识别异常。然而,现有方法主要依赖将3D观测投影到多视图表示中,这些表示主要捕捉几何线索而非现实视觉语义,并通过在RGB数据上预训练的视觉编码器处理,导致编码器与投影表示之间存在显著的领域差距。为解决此问题,我们提出Align3D-AD,一个统一的两阶段框架,利用辅助类别的RGB模态作为跨模态指导进行零样本3D异常检测。首先,我们引入了跨模态特征对齐范式,将渲染特征映射到RGB语义空间。不同于以往依赖预训练编码器的隐式方法,我们的方法实现了从RGB观测到语义的直接转移。进一步引入语义一致性重加权策略,通过根据整体语义一致性重新加权局部区域来优化特征对齐。其次,我们提出一种模态感知的提示学习框架,具有双提示对比对齐。通过为RGB对齐和渲染特征分配独立提示,我们的方法捕捉了跨模态的互补语义,而对比对齐进一步增强了提示表示以提高判别性。在MVTec3D-AD、Eyecandies和Real3D-AD等数据集上的广泛实验表明,Align3D-AD在单对多和跨数据集设置下均优于现有零样本方法,突显了其泛化能力和鲁棒性。代码和数据集将在论文被接受后公开。

英文摘要:

Zero-shot 3D anomaly detection aims to identify anomalies without access to training data from target categories. However, existing methods mainly rely on projecting 3D observations into multi-view representations that primarily capture geometric cues rather than realistic visual semantics and process them with vision encoders pretrained on RGB data, leading to a significant domain gap between the encoder and the projected representations. To address this issue, we propose Align3D-AD, a unified two-stage framework that leverages the RGB modality from auxiliary categories as cross-modal guidance for zero-shot 3D anomaly detection. First, we introduce a cross-modal feature alignment paradigm that maps rendering features into the RGB semantic space. Unlike prior works that implicitly rely on pretrained encoders, our method enables direct semantic transfer from RGB observations. A semantic consistency reweighting strategy is further introduced to refine feature alignment by reweighting local regions according to holistic semantic consistency. Second, we propose a modality-aware prompt learning framework with dual-prompt contrastive alignment. By assigning independent prompts to RGB-aligned and rendering features, our method captures complementary semantics across modalities, while the contrastive alignment further enhances prompt representations to improve discriminability. Extensive experiments on MVTec3D-AD, Eyecandies, and Real3D-AD demonstrate that Align3D-AD consistently outperforms existing zero-shot methods under both one-vs-rest and cross-dataset settings, highlighting its generalization capability and robustness. Code and the dataset will be made available once our paper is accepted.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑