arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2106.11310cs.CV
收藏

走向长格式视频理解

Towards Long-Form Video Understanding

The University of Texas at Austin(德克萨斯大学奥斯汀分校)

Chao-Yuan Wu, Philipp Krähenbühl

摘要:

我们的世界提供了一个永无止境的视觉刺激流,但今天的视觉系统只能在几秒钟内准确地识别模式。这些系统理解现在,但无法将其置于过去或未来事件的背景中。本文主要研究长格式视频理解。我们介绍了一个框架,用于建模长格式的视频和开发大规模数据集上的评估协议。我们发现,现有的国家的最先进的短期模型是有限的长期任务。一种新的以对象为中心的基于transformer的视频识别架构在7个不同的任务上表现得更好。它在AVA数据集上的表现也优于同类最新技术水平。

英文摘要:

Our world offers a never-ending stream of visual stimuli, yet today's vision systems only accurately recognize patterns within a few seconds. These systems understand the present, but fail to contextualize it in past or future events. In this paper, we study long-form video understanding. We introduce a framework for modeling long-form videos and develop evaluation protocols on large-scale datasets. We show that existing state-of-the-art short-term models are limited for long-form tasks. A novel object-centric transformer-based video recognition architecture performs significantly better on 7 diverse tasks. It also outperforms comparable state-of-the-art on the AVA dataset.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑