Live-E2T:通过去重事件推理与思维链实现视频中的实时威胁监控
Live-E2T: Real-time Threat Monitoring in Video via Deduplicated Event Reasoning and Chain-of-Thought
AI总结:
Live-E2T框架通过语义元组表示、在线事件去重更新和思维链微调大语言模型,在实时视频威胁监控中同时提升检测准确率、实时性与可解释性。
摘要:
实时威胁监控旨在识别视频流中的威胁行为,并通过解释性文本提供对威胁事件的推理与评估。然而,现有主流方法,无论是基于监督学习还是生成模型,都难以同时满足实时性能与决策可解释性的严苛要求。为弥合这一差距,我们提出了Live-E2T,一个通过三种协同机制统一上述两个目标的新框架。首先,我们将视频帧解构为结构化的人-物-交互-场所语义元组。该方法创建了一种紧凑且语义聚焦的表示,规避了传统特征压缩中常见的信息退化问题。其次,提出了一种高效的在线事件去重与更新机制,用于过滤时空冗余,从而确保系统的实时响应能力。最后,我们采用思维链(Chain-of-Thought)策略微调一个大语言模型,使其具备对事件序列进行透明且逻辑化推理的能力,以生成连贯的威胁评估报告。在包括XD-Violence和UCF-Crime在内的基准数据集上的大量实验表明,Live-E2T在威胁检测准确率、实时效率以及至关重要的可解释性维度上均显著优于当前最先进的方法。
英文摘要:
Real-time threat monitoring identifies threatening behaviors in video streams and provides reasoning and assessment of threat events through explanatory text. However, prevailing methodologies, whether based on supervised learning or generative models, struggle to concurrently satisfy the demanding requirements of real-time performance and decision explainability. To bridge this gap, we introduce Live-E2T, a novel framework that unifies these two objectives through three synergistic mechanisms. First, we deconstruct video frames into structured Human-Object-Interaction-Place semantic tuples. This approach creates a compact, semantically focused representation, circumventing the information degradation common in conventional feature compression. Second, an efficient online event deduplication and updating mechanism is proposed to filter spatio-temporal redundancies, ensuring the system's real time responsiveness. Finally, we fine-tune a Large Language Model using a Chain-of-Thought strategy, endow it with the capability for transparent and logical reasoning over event sequences to produce coherent threat assessment reports. Extensive experiments on benchmark datasets, including XD-Violence and UCF-Crime, demonstrate that Live-E2T significantly outperforms state-of-the-art methods in terms of threat detection accuracy, real-time efficiency, and the crucial dimension of explainability.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。