arXiv 2412.202012025-09-23cs.CVcs.AI

将可解释性与轻量化设计注入弱监督视频异常检测系统

Injecting Explainability and Lightweight Design into Weakly Supervised Video Anomaly Detection Systems

Tamkang University（淡江大学）；The University of Melbourne（墨尔本大学）；National Institute of Technology, Shillong（印度国立技术学院西隆分校）

Wen-Dong Jiang, Chih-Yung Chang, Hsiang-Chuan Chang, Ji-Yuan Chen, Diptendu Sinha Roy

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出两阶段跨模态视频异常检测系统TCVADS，通过知识蒸馏实现轻量化粗分类，并利用CLIP跨模态对比学习进行细粒度可解释分类，在边缘设备上显著提升检测性能与效率。

摘要：

弱监督监控异常检测（WSMAD）利用弱监督学习来识别异常，这是智慧城市监控中的一项关键任务。然而，现有的多模态方法由于复杂度较高，往往难以满足边缘设备对实时性和可解释性的要求。本文提出了TCVADS（两阶段跨模态视频异常检测系统），该系统利用知识蒸馏和跨模态对比学习，在边缘设备上实现高效、准确且可解释的异常检测。TCVADS分两个阶段运行：粗粒度快速分类和细粒度详细分析。在第一阶段，TCVADS从视频帧中提取特征并将其输入时间序列分析模块，该模块充当教师模型。随后通过知识蒸馏将知识迁移到一个简化的卷积网络（学生模型）中，用于二分类。一旦检测到异常，第二阶段被触发，采用细粒度多类分类模型。该阶段利用CLIP进行文本与图像的跨模态对比学习，增强可解释性，并通过专门设计的三元文本关系实现精细分类。实验结果表明，TCVADS在模型性能、检测效率和可解释性方面显著优于现有方法，为智慧城市监控应用提供了有价值的贡献。

英文摘要：

Weakly Supervised Monitoring Anomaly Detection (WSMAD) utilizes weak supervision learning to identify anomalies, a critical task for smart city monitoring. However, existing multimodal approaches often fail to meet the real-time and interpretability requirements of edge devices due to their complexity. This paper presents TCVADS (Two-stage Cross-modal Video Anomaly Detection System), which leverages knowledge distillation and cross-modal contrastive learning to enable efficient, accurate, and interpretable anomaly detection on edge devices.TCVADS operates in two stages: coarse-grained rapid classification and fine-grained detailed analysis. In the first stage, TCVADS extracts features from video frames and inputs them into a time series analysis module, which acts as the teacher model. Insights are then transferred via knowledge distillation to a simplified convolutional network (student model) for binary classification. Upon detecting an anomaly, the second stage is triggered, employing a fine-grained multi-class classification model. This stage uses CLIP for cross-modal contrastive learning with text and images, enhancing interpretability and achieving refined classification through specially designed triplet textual relationships. Experimental results demonstrate that TCVADS significantly outperforms existing methods in model performance, detection efficiency, and interpretability, offering valuable contributions to smart city monitoring applications.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。