重新思考高效通用事件边界检测的架构设计
Rethinking the Architecture Design for Efficient Generic Event Boundary Detection
AI总结：
本文通过重新审视GEBD模型架构，发现图像域骨干网络存在冗余和干扰问题，提出使用视频域骨干网络联合时空建模的EfficientGEBD系列模型，在性能上最高提升1.7%，速度提升280%。
摘要：
通用事件边界检测（GEBD）受人类视觉认知行为的启发，即人类会持续将视频分割成有意义的时间片段，在视频编辑等各类应用中具有实用价值。在本文中，我们证明最先进的GEBD模型往往优先考虑最终性能而非模型复杂度，导致推理速度低，阻碍了其在真实场景中的高效部署。我们通过实验重新审视GEBD模型的架构，并发现了若干令人惊讶的结果，为应对这一挑战做出贡献。首先，我们揭示了一个简洁的GEBD基线模型无需任何复杂设计即可达到有竞争力的性能。其次，我们发现GEBD模型中广泛应用的图像域骨干网络可能包含大量架构冗余，这促使我们逐步“现代化”每个组件以提高效率。第三，我们表明使用图像域骨干网络以“先空间后时间”的贪心方式进行时空学习的GEBD模型可能遭受干扰问题，这可能是GEBD效率低下的根源。使用视频域骨干网络联合进行时空建模是解决该问题的有效方案。我们探索的成果是一系列名为EfficientGEBD的GEBD模型，在相同骨干网络下，其性能比之前的最先进方法最高提升1.7%，速度提升280%。我们的研究促使社区在设计现代GEBD方法时考虑模型复杂度，特别是在资源受限的应用中。代码可在https://github.com/Ziwei-Zheng/EfficientGEBD获取。
英文摘要：
Generic event boundary detection (GEBD), inspired by human visual cognitive behaviors of consistently segmenting videos into meaningful temporal chunks, finds utility in various applications such as video editing and. In this paper, we demonstrate that SOTA GEBD models often prioritize final performance over model complexity, resulting in low inference speed and hindering efficient deployment in real-world scenarios. We contribute to addressing this challenge by experimentally reexamining the architecture of GEBD models and uncovering several surprising findings. Firstly, we reveal that a concise GEBD baseline model already achieves promising performance without any sophisticated design. Secondly, we find that the widely applied image-domain backbones in GEBD models can contain plenty of architecture redundancy, motivating us to gradually ``modernize'' each component to enhance efficiency. Thirdly, we show that the GEBD models using image-domain backbones conducting the spatiotemporal learning in a spatial-then-temporal greedy manner can suffer from a distraction issue, which might be the inefficient villain for GEBD. Using a video-domain backbone to jointly conduct spatiotemporal modeling is an effective solution for this issue. The outcome of our exploration is a family of GEBD models, named EfficientGEBD, significantly outperforms the previous SOTA methods by up to 1.7\% performance gain and 280\% speedup under the same backbone. Our research prompts the community to design modern GEBD methods with the consideration of model complexity, particularly in resource-aware applications. The code is available at \url{https://github.com/Ziwei-Zheng/EfficientGEBD}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
