通过文本引导增强弱监督多模态视频异常检测
Enhancing Weakly Supervised Multimodal Video Anomaly Detection through Text Guidance
AI总结：
本文提出文本引导框架，通过多阶段文本增强和多尺度瓶颈Transformer融合模块，提升弱监督多模态视频异常检测的性能。
摘要：
弱监督多模态视频异常检测引起了广泛关注，但文本模态的潜力仍待深入探索。文本提供了明确的语义信息，可以增强异常特征描述并减少误报。然而，提取有效的文本特征具有挑战性，因为通用语言模型无法捕捉到异常特定的细微差别，且相关描述稀缺。此外，多模态融合常面临冗余和不平衡的问题。为了解决这些问题，我们提出了一种新的文本引导框架。首先，我们引入基于上下文学习的多阶段文本增强机制，生成高质量的异常文本样本以微调文本特征提取器。其次，我们设计了多尺度瓶颈Transformer融合模块，利用压缩瓶颈标记逐步整合跨模态信息，缓解冗余和不平衡。在UCF-Crime和XD-Violence上的实验表明取得了最先进的性能。
英文摘要：
Weakly supervised multimodal video anomaly detection has gained significant attention, yet the potential of the text modality remains under-explored. Text provides explicit semantic information that can enhance anomaly characterization and reduce false alarms. However, extracting effective text features is challenging due to the inability of general-purpose language models to capture anomaly-specific nuances and the scarcity of relevant descriptions. Furthermore, multimodal fusion often suffers from redundancy and imbalance. To address these issues, we propose a novel text-guided framework. First, we introduce an in-context learning-based multi-stage text augmentation mechanism to generate high-quality anomaly text samples for fine-tuning the text feature extractor. Second, we design a multi-scale bottleneck Transformer fusion module that uses compressed bottleneck tokens to progressively integrate information across modalities, mitigating redundancy and imbalance. Experiments on UCF-Crime and XD-Violence demonstrate state-of-the-art performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
