arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.07772cs.CV
收藏

ESOM:基于开放世界动态定义的高效流视频异常理解

ESOM: Efficiently Understanding Streaming Video Anomalies with Open-world Dynamic Definitions

State Key Laboratory of Media Convergence and Communication, Communication University of China(媒体融合与传播国家重点实验室,中国传媒大学)

Zihao Liu, Xiaoyu Wu, Wenna Li, Jianqin Wu, Linlin Yang

AI总结:

本文提出ESOM模型,通过定义规范化、帧间匹配、混合流内存和概率评分模块,实现高效流视频异常检测,同时引入OpenDef-Bench基准测试,在单GPU上实现实时效率和先进性能。

摘要:

开放世界视频异常检测(OWVAD)旨在在不同异常定义下检测和解释异常事件,对智能监控和直播内容审核等应用至关重要。最近的基于大规模语言模型(MLLM)的方法在开放世界泛化方面表现出色,但仍然面临三个主要限制:部署效率低、缺乏流处理适应性以及在建模和评估中对动态异常定义的支持有限。为了解决这些问题,本文提出了ESOM,一种高效的流OWVAD模型,该模型以无训练方式运行。ESOM包括一个定义规范化模块,用于结构化用户提示以减少幻觉,一个帧间匹配帧内令牌合并模块,用于压缩冗余视觉令牌,一个混合流内存模块用于高效因果推理,以及一个概率评分模块,将区间级文本输出转换为帧级异常评分。此外,本文引入了OpenDef-Bench,一个包含干净监控视频和多样自然异常定义的新基准,用于在不同条件下评估性能。广泛实验表明,ESOM在单个GPU上实现了实时效率,并在异常时间定位、分类和描述生成方面取得了最先进的性能。代码和基准将发布在https://github.com/Kamino666/ESOM_OpenDef-Bench。

英文摘要:

Open-world video anomaly detection (OWVAD) aims to detect and explain abnormal events under different anomaly definitions, which is important for applications such as intelligent surveillance and live-streaming content moderation. Recent MLLM-based methods have shown promising open-world generalization, but still suffer from three major limitations: inefficiency for practical deployment, lack of streaming processing adaptation, and limited support for dynamic anomaly definitions in both modeling and evaluation. To address these issues, this paper proposes ESOM, an efficient streaming OWVAD model that operates in a training-free manner. ESOM includes a Definition Normalization module to structure user prompts for reducing hallucination, an Inter-frame-matched Intra-frame Token Merging module to compress redundant visual tokens, a Hybrid Streaming Memory module for efficient causal inference, and a Probabilistic Scoring module that converts interval-level textual outputs into frame-level anomaly scores. In addition, this paper introduces OpenDef-Bench, a new benchmark with clean surveillance videos and diverse natural anomaly definitions for evaluating performance under varying conditions. Extensive experiments show that ESOM achieves real-time efficiency on a single GPU and state-of-the-art performance in anomaly temporal localization, classification, and description generation. The code and benchmark will be released at https://github.com/Kamino666/ESOM_OpenDef-Bench.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑