DeepSVU: 向深入的安全导向视频理解迈进：通过统一的物理世界正则化MoE
DeepSVU: Towards In-depth Security-oriented Video Understanding via Unified Physical-world Regularized MoE
AI总结：
本文提出DeepSVU任务，通过统一物理世界正则化MoE方法，提升视频中威胁的识别、归因和评估能力。
摘要：
在文献中，针对安全导向视频理解（SVU）的先前研究主要集中在检测和定位威胁（例如枪击、抢劫）在视频中，而缺乏有效生成和评估威胁原因的能力。受这些差距的启发，本文引入了一个新的聊天范式SVU任务，即深入的安全导向视频理解（DeepSVU），旨在不仅识别和定位威胁，还归因和评估威胁片段的原因。此外，本文揭示了所提出任务的两个关键挑战：1）如何有效建模粗到细的物理世界信息（例如人类行为、物体交互和背景上下文）以提升DeepSVU任务；2）如何自适应地权衡这些因素。为了解决这些挑战，本文提出了一种新的统一物理世界正则化MoE（UPRM）方法。具体来说，UPRM结合了两个关键组件：统一物理世界增强MoE（UPE）块和物理世界权衡正则器（PTR），分别解决上述两个挑战。在我们的DeepSVU指令数据集（即UCF-C指令和CUVA指令）上进行了广泛的实验，结果表明UPRM在几个先进的视频大语言模型以及非VLM方法上均表现出色。这些结果证明了粗到细的物理世界信息在DeepSVU任务中的重要性，并展示了我们的UPRM在捕捉此类信息方面的有效性。
英文摘要：
In the literature, prior research on Security-oriented Video Understanding (SVU) has predominantly focused on detecting and localize the threats (e.g., shootings, robberies) in videos, while largely lacking the effective capability to generate and evaluate the threat causes. Motivated by these gaps, this paper introduces a new chat paradigm SVU task, i.e., In-depth Security-oriented Video Understanding (DeepSVU), which aims to not only identify and locate the threats but also attribute and evaluate the causes threatening segments. Furthermore, this paper reveals two key challenges in the proposed task: 1) how to effectively model the coarse-to-fine physical-world information (e.g., human behavior, object interactions and background context) to boost the DeepSVU task; and 2) how to adaptively trade off these factors. To tackle these challenges, this paper proposes a new Unified Physical-world Regularized MoE (UPRM) approach. Specifically, UPRM incorporates two key components: the Unified Physical-world Enhanced MoE (UPE) Block and the Physical-world Trade-off Regularizer (PTR), to address the above two challenges, respectively. Extensive experiments conduct on our DeepSVU instructions datasets (i.e., UCF-C instructions and CUVA instructions) demonstrate that UPRM outperforms several advanced Video-LLMs as well as non-VLM approaches. Such information.These justify the importance of the coarse-to-fine physical-world information in the DeepSVU task and demonstrate the effectiveness of our UPRM in capturing such information.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
