arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.04274cs.CV
收藏

用于通用事件边界检测的细粒度动态网络

Fine-grained Dynamic Network for Generic Event Boundary Detection

Ziwei Zheng, Lijun He, Le Yang, Fan Li

AI总结:

本文提出动态流水线 DyBDet,通过多出口网络和多阶差分检测器实现通用事件边界的自适应细粒度检测,在 Kinetics-GEBD 和 TAPOS 上显著提升性能与效率。

摘要:

通用事件边界检测(GEBD)旨在精确定位人类自然感知到的事件边界,在理解长视频中发挥着关键作用。由于通用边界具有多样化的特性,涵盖不同的视频外观、物体和动作,该任务仍然具有挑战性。现有方法通常采用相同的处理流程来检测各种边界,而不考虑它们各自独特的特征和检测难度,导致性能欠佳。从直觉上看,一种更智能、更合理的方式是通过考虑边界的特殊属性来自适应地检测边界。鉴于此,我们提出了一种新颖的用于通用事件边界的动态流水线,命名为 DyBDet。通过引入多出口网络架构,DyBDet 自动学习不同视频片段的子网络分配,从而实现对各种边界的细粒度检测。此外,我们还提出了一种多阶差分检测器,以确保通用边界能够被有效识别和自适应处理。在具有挑战性的 Kinetics-GEBD 和 TAPOS 数据集上进行的大量实验表明,采用动态策略显著有益于 GEBD 任务,与当前最先进的方法相比,在性能和效率方面都带来了明显的提升。

英文摘要:

Generic event boundary detection (GEBD) aims at pinpointing event boundaries naturally perceived by humans, playing a crucial role in understanding long-form videos. Given the diverse nature of generic boundaries, spanning different video appearances, objects, and actions, this task remains challenging. Existing methods usually detect various boundaries by the same protocol, regardless of their distinctive characteristics and detection difficulties, resulting in suboptimal performance. Intuitively, a more intelligent and reasonable way is to adaptively detect boundaries by considering their special properties. In light of this, we propose a novel dynamic pipeline for generic event boundaries named DyBDet. By introducing a multi-exit network architecture, DyBDet automatically learns the subnet allocation to different video snippets, enabling fine-grained detection for various boundaries. Besides, a multi-order difference detector is also proposed to ensure generic boundaries can be effectively identified and adaptively processed. Extensive experiments on the challenging Kinetics-GEBD and TAPOS datasets demonstrate that adopting the dynamic strategy significantly benefits GEBD tasks, leading to obvious improvements in both performance and efficiency compared to the current state-of-the-art.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑