arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1804.09066cs.CVcs.AIcs.IRcs.MM
收藏

ECO:用于在线视频理解的高效卷积网络

ECO: Efficient Convolutional Network for Online Video Understanding

University of Freiburg(弗莱堡大学)

Mohammadreza Zolfaghari, Kamaljeet Singh, Thomas Brox

摘要:

视频理解的现有技术存在两个问题:(1)推理的主要部分在视频中本地执行,因此,它错过了跨越几秒的动作内的重要关系。(2)虽然存在具有快速每帧处理的局部方法,但是对整个视频的处理效率不高,并且阻碍了快速视频检索或长期活动的在线分类。在本文中,我们介绍了一种网络架构,该架构将长期内容考虑在内,并同时实现快速的逐视频处理。该架构是基于合并长期的内容已经在网络中,而不是在一个事后的融合。与利用相邻帧在很大程度上是冗余的采样策略一起,这产生了高达每秒230个视频的高质量动作分类和视频描述,其中每个视频可以由几百帧组成。该方法在所有数据集上都具有竞争力的性能,同时比最先进的方法快10倍到80倍。

英文摘要:

The state of the art in video understanding suffers from two problems: (1) The major part of reasoning is performed locally in the video, therefore, it misses important relationships within actions that span several seconds. (2) While there are local methods with fast per-frame processing, the processing of the whole video is not efficient and hampers fast video retrieval or online classification of long-term activities. In this paper, we introduce a network architecture that takes long-term content into account and enables fast per-video processing at the same time. The architecture is based on merging long-term content already in the network rather than in a post-hoc fusion. Together with a sampling strategy, which exploits that neighboring frames are largely redundant, this yields high-quality action classification and video captioning at up to 230 videos per second, where each video can consist of a few hundred frames. The approach achieves competitive performance across all datasets while being 10x to 80x faster than state-of-the-art methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑