arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2107.07089cs.CVcs.AI
收藏

STAR:基于稀疏变换器的动作识别

STAR: Sparse Transformer-based Action Recognition

University of California Los Angeles(加利福尼亚大学洛杉矶分校);The Pennsylvania State University(宾夕法尼亚州立大学);Stevens Institute of Technology(斯蒂文斯理工学院)

Feng Shi, Chonghan Lee, Liang Qiu, Yizhou Zhao, Tianyi Shen, Shivran Muralidhar, Tian Han, Song-Chun Zhu, Vijaykrishnan Narayanan

摘要:

人类行动和行为的认知系统已经发展成为一种深度学习机制,特别是近年来图卷积网络的出现改变了该领域。然而,以前的工作主要集中在基于稠密图卷积网络的过参数化和复杂模型上,导致训练和推理效率低下。与此同时,基于Transformer架构的模型尚未在人类动作和行为估计的认知应用中得到很好的探索。本文提出了一种新的基于小波变换的人体动作识别模型,该模型在空间维度上具有稀疏注意力,在时间维度上具有分段的线性注意力。我们的模型还可以处理分组为单个批次的可变长度的视频剪辑。实验表明,我们的模型可以达到相当的性能,而使用更少的可训练参数,并实现高速度的训练和推理。实验结果表明,与基线模型相比,该模型在具有竞争力的精度下,获得了4~ 18倍的加速比和1/7~1/15的模型大小。

英文摘要:

The cognitive system for human action and behavior has evolved into a deep learning regime, and especially the advent of Graph Convolution Networks has transformed the field in recent years. However, previous works have mainly focused on over-parameterized and complex models based on dense graph convolution networks, resulting in low efficiency in training and inference. Meanwhile, the Transformer architecture-based model has not yet been well explored for cognitive application in human action and behavior estimation. This work proposes a novel skeleton-based human action recognition model with sparse attention on the spatial dimension and segmented linear attention on the temporal dimension of data. Our model can also process the variable length of video clips grouped as a single batch. Experiments show that our model can achieve comparable performance while utilizing much less trainable parameters and achieve high speed in training and inference. Experiments show that our model achieves 4~18x speedup and 1/7~1/15 model size compared with the baseline models at competitive accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑