释放反向蒸馏在异常检测中的潜力
Unlocking the Potential of Reverse Distillation for Anomaly Detection
AI总结：
针对反向蒸馏异常检测中教师特征区分不足、学生无异常生成能力弱及细节丢失问题，提出带专家网络的反向蒸馏框架与引导信息注入机制，在多基准上取得更优性能，释放反向蒸馏潜力。
摘要：
知识蒸馏（KD）是一种颇具前景的无监督异常检测（AD）方法。然而，学生网络的过泛化往往会削弱教师与学生在异常区域的关键表征差异，导致检测失败。为解决这一问题，被广泛认可的反向蒸馏（RD）范式设计了非对称的教师与学生结构，以编码器作为教师、解码器作为学生。但反向蒸馏的设计无法确保教师编码器有效区分正常与异常特征，也无法保证学生解码器生成无异常特征。此外，跳跃连接的缺失会导致特征重建过程中细节信息丢失。为解决这些问题，我们提出了带专家的反向蒸馏（RD with Expert），引入一种新颖的专家-教师-学生网络，可同时对教师编码器和学生解码器进行蒸馏。新增的专家网络增强了学生生成正常特征的能力，优化了教师对正常与异常特征的区分度，减少漏检。此外，我们设计了引导信息注入模块，用于过滤教师特征并将其传递给学生，改善细节重建效果，降低假阳性。在多个基准上的实验证明，我们的方法在反向蒸馏范式下优于现有无监督异常检测方法，充分释放了反向蒸馏的潜力。
英文摘要：
Knowledge Distillation (KD) is a promising approach for unsupervised Anomaly Detection (AD). However, the student network's over-generalization often diminishes the crucial representation differences between teacher and student in anomalous regions, leading to detection failures. To addresses this problem, the widely accepted Reverse Distillation (RD) paradigm designs the asymmetry teacher and student, using an encoder as teacher and a decoder as student. Yet, the design of RD does not ensure that the teacher encoder effectively distinguishes between normal and abnormal features or that the student decoder generates anomaly-free features. Additionally, the absence of skip connections results in a loss of fine details during feature reconstruction. To address these issues, we propose RD with Expert, which introduces a novel Expert-Teacher-Student network for simultaneous distillation of both the teacher encoder and student decoder. The added expert network enhances the student's ability to generate normal features and optimizes the teacher's differentiation between normal and abnormal features, reducing missed detections. Additionally, Guided Information Injection is designed to filter and transfer features from teacher to student, improving detail reconstruction and minimizing false positives. Experiments on several benchmarks prove that our method outperforms existing unsupervised AD methods under RD paradigm, fully unlocking RD's potential.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
