arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2110.05904cs.CVeess.IV
收藏

视频即图:用于视频动作识别的结构化图模型

Video Is Graph: Structured Graph Module for Video Action Recognition

School of Artificial Intelligence and Computer Science(人工智能与计算机科学学院)

Rongchang Li, Xiao-Jun Wu, Tianyang Xu

摘要:

在动作识别领域,视频片段通常被视为有序帧进行后续处理。为了实现时空感知,现有的方法提出在卷积层中嵌入相邻的时间交互。因此,可以通过分层堆叠多个局部层来获得全局语义信息。然而,这种全局时间累积只能反映深层的高层语义,忽略了浅层潜在的低层整体线索。在本文中,我们首先提出将视频序列转换成一个图,以获得时间帧之间的直接长期依赖关系。为了在转换过程中保留顺序信息,我们设计了一个结构化图模块(SGM),在整个网络中实现细粒度的时间交互。特别地,SGM将每个节点的邻居划分为多个时间区域,以便提取具有不同序列流的全局结构信息。在标准基准数据集上进行了广泛的实验,即,Something-Something V1 & V2、Diving 48、Kinetics-400、UCF 101和HMDB 51。实验结果表明,SGM算法能够以较小的计算复杂性达到较高的精度.

英文摘要:

In the field of action recognition, video clips are always treated as ordered frames for subsequent processing. To achieve spatio-temporal perception, existing approaches propose to embed adjacent temporal interaction in the convolutional layer. The global semantic information can therefore be obtained by stacking multiple local layers hierarchically. However, such global temporal accumulation can only reflect the high-level semantics in deep layers, neglecting the potential low-level holistic clues in shallow layers. In this paper, we first propose to transform a video sequence into a graph to obtain direct long-term dependencies among temporal frames. To preserve sequential information during transformation, we devise a structured graph module (SGM), achieving fine-grained temporal interactions throughout the entire network. In particular, SGM divides the neighbors of each node into several temporal regions so as to extract global structural information with diverse sequential flows. Extensive experiments are performed on standard benchmark datasets, i.e., Something-Something V1 & V2, Diving48, Kinetics-400, UCF101, and HMDB51. The reported performance and analysis demonstrate that SGM can achieve outstanding precision with less computational complexity.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑