arXiv 2412.089492026-07-08cs.CV

调谐反向蒸馏：使用跨模态调谐器增强多模态工业异常检测

Tuned Reverse Distillation: Enhancing Multimodal Industrial Anomaly Detection with Crossmodal Tuners

School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）；Key Laboratory of Intelligent Bionic Unmanned Systems, Ministry of Education（教育部智能仿生无人系统重点实验室）；School of Artificial Intelligence, University of Science and Technology Beijing（北京科技大学人工智能学院）；School of Computer Science and Technology, Beijing Jiaotong University（北京交通大学计算机科学与技术学院）

Xinyue Liu, Jianyuan Wang, Biao Leng, Shuo Zhang

备注：: Accepted by TMM

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文针对多模态工业异常检测问题，提出基于多分支设计的调谐反向蒸馏方法，通过为各模态设独立分支及设计跨模态调谐器，增强模态交互，能更精细检测异常，实验验证其在多模态异常检测和定位上达最优性能。

摘要：

知识蒸馏（KD）在无监督图像异常检测（AD）中已得到广泛研究，但其在无监督多模态AD中的应用仍未得到充分探索。现有的基于KD的多模态AD方法使用融合的多模态特征来获得教师表示面临挑战。仅存在于一种模态中的异常可能无法在融合的教师特征中有效捕获，导致检测失败。此外，这些方法没有充分利用对有效异常检测至关重要的丰富的模态内和模态间信息。在本文中，我们提出基于多分支设计的调谐反向蒸馏（TRD）来实现多模态工业AD。通过为每个模态分配独立分支，我们的方法能够更精细地检测每个模态内的异常。此外，我们通过设计包括跨模态滤波器和放大器的两个跨模态调谐器来增强蒸馏过程中的模态间交互。借助跨模态映射的思想，学生网络能够更好地学习正常特征，同时确保有效检测所有模态中的异常。在多模态AD数据集上的实验验证表明，我们的方法在多模态异常检测和定位方面取得了当前最优的性能。代码可在该https网址获取。

英文摘要：

Knowledge distillation (KD) has been widely studied in unsupervised image Anomaly Detection (AD), but its application to unsupervised multimodal AD remains underexplored. Existing KD-based methods for multimodal AD that use fused multimodal features to obtain teacher representations face challenges. Anomalies that only exist in one modality may not be effectively captured in the fused teacher features, leading to detection failures. Besides, these methods do not fully leverage the rich intra- and inter-modality information that are critical for effective anomaly detection. In this paper, we propose Tuned Reverse Distillation (TRD) based on Multi-branch design to realize Multimodal Industrial AD. By assigning independent branches to each modality, our method enables finer detection of anomalies within each modality. Furthermore, we enhance the interaction between modalities during the distillation process by designing two Crossmodal Tuners including Crossmodal Filter and Amplifier. With the idea of crossmodal mapping, the student network is allowed to better learn normal features while anomalies in all modalities are ensured to be effectively detected. Experimental verifications on multimodal AD datasets demonstrate that our method achieves state-of-the-art performance in multimodal anomaly detection and localization. Code is available at https://github.com/hito2448/TRD.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。