从架构视角重新审视用于3D异常检测的多模态融合
Revisiting Multimodal Fusion for 3D Anomaly Detection from an Architectural Perspective
AI总结：
本文系统研究了多模态融合架构设计对3D异常检测的影响，涵盖模块内和模块间融合层级，并提出3D-ADNAS首次同时搜索多模态融合策略与模态特定模块，在准确率、帧率和内存上取得一致提升且在少样本任务中潜力显著。
摘要：
提升3D异常检测（3D-AD）多模态融合的现有工作主要集中在设计更有效的多模态融合策略上。然而，很少有人关注分析多模态融合架构（拓扑）设计对3D-AD的贡献。本文旨在弥补这一空白，系统研究多模态融合架构设计对3D-AD的影响。本工作在模块内融合层级考虑多模态融合架构设计，即独立的模态特定模块，涉及具有特定融合操作的早期、中期或晚期多模态特征，并在模块间融合层级考虑，即融合这些模块的策略。在这两种情况下，我们首先通过理论和实验探索架构设计如何影响3D-AD来得出见解。然后，我们扩展了SOTA神经架构搜索（NAS）范式，并首次提出3D-ADNAS，以同时搜索多模态融合策略和模态特定模块。大量实验表明，3D-ADNAS在准确率、帧率和内存使用方面，在各种模型容量下均获得了3D-AD的一致改进，并在处理少样本3D-AD任务中展现出巨大潜力。
英文摘要：
Existing efforts to boost multimodal fusion of 3D anomaly detection (3D-AD) primarily concentrate on devising more effective multimodal fusion strategies. However, little attention was devoted to analyzing the role of multimodal fusion architecture (topology) design in contributing to 3D-AD. In this paper, we aim to bridge this gap and present a systematic study on the impact of multimodal fusion architecture design on 3D-AD. This work considers the multimodal fusion architecture design at the intra-module fusion level, i.e., independent modality-specific modules, involving early, middle or late multimodal features with specific fusion operations, and also at the inter-module fusion level, i.e., the strategies to fuse those modules. In both cases, we first derive insights through theoretically and experimentally exploring how architectural designs influence 3D-AD. Then, we extend SOTA neural architecture search (NAS) paradigm and propose 3D-ADNAS to simultaneously search across multimodal fusion strategies and modality-specific modules for the first time.Extensive experiments show that 3D-ADNAS obtains consistent improvements in 3D-AD across various model capacities in terms of accuracy, frame rate, and memory usage, and it exhibits great potential in dealing with few-shot 3D-AD tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
