基于文本引导的细粒度视频异常理解
Text-guided Fine-Grained Video Anomaly Understanding
AI总结：
本文提出T-VAU框架，通过多模态推理提取细粒度视频异常证据，结合Anomaly Heatmap Decoder和Region-aware Anomaly Encoder提升异常检测与解释能力。
摘要：
视频中微妙异常事件常表现为弱时空线索，传统检测系统难以捕捉。现有方法提供粗略二元判断，而大视觉-语言模型虽能生成文本判断但缺乏精确定位。为此，我们提出Text-guided Fine-Grained Video Anomaly Understanding T-VAU框架，通过多模态推理提取像素级时空异常热图。引入Anomaly Heatmap Decoder进行视觉-文本特征对齐，生成中间视觉表征的像素级时空异常热图。进一步设计Region-aware Anomaly Encoder将这些热图转换为结构化提示嵌入，使大视觉-语言模型在统一推理流程中实现异常检测、定位和语义解释。构建目标级细粒度视频-文本异常数据集，支持细粒度监督。实验表明，T-VAU在基准测试中显著提升异常定位和文本推理性能，取得BLEU-4指标强结果及Yes/No决策准确率，提供可解释的像素级时空证据。代码将在https://github.com/momiji-bit/T-VAU发布。
英文摘要：
Subtle abnormal events in videos often manifest as weak spatio-temporal cues that are easily overlooked by conventional anomaly detection systems. Existing video anomaly detection approaches typically provide coarse binary anomaly decisions without interpretable evidence, while large vision-language models (LVLMs) can produce textual judgments but lack precise localization of subtle visual signals. To address this gap, we propose Text-guided Fine-Grained Video Anomaly Understanding T-VAU, a framework that grounds subtle anomaly evidence into multimodal reasoning. Specifically, we introduce an Anomaly Heatmap Decoder (AHD) that performs visual-textual feature alignment to extract pixel-level spatio-temporal anomaly heatmaps from intermediate visual representations. We further design a Region-aware Anomaly Encoder (RAE) that converts these heatmaps into structured prompt embeddings, enabling the LVLM to perform anomaly detection, localization, and semantic explanation in a unified reasoning pipeline. To support fine-grained supervision, we construct a target-level fine-grained video-text anomaly dataset derived from ShanghaiTech and UBnormal with detailed annotations of object appearance, localization, and motion trajectories. Extensive experiments demonstrate that T-VAU significantly improves anomaly localization and textual reasoning performance on both benchmarks, achieving strong results in BLEU-4 metrics and Yes/No decision accuracy while providing interpretable pixel-level spatio-temporal evidence for anomaly understanding. The code will be available at https://github.com/momiji-bit/T-VAU.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
