arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2008.01057cs.CV
收藏

基于有效伪三维CNN的人体动作识别

Residual Frames with Efficient Pseudo-3D CNN for Human Action Recognition

InnoPeak Technology(InnoPeak科技)

Jiawei Chen, Jenson Hsiao, Chiu Man Ho

摘要:

人类动作识别被认为是监控或视频理解等领域的关键基石。尽管最近在基于视频的动作识别的端到端解决方案的开发方面取得了进展,但实现最先进的性能仍然需要使用辅助手工制作的运动表示,例如,光流,这通常需要计算。在这项工作中,我们提出使用残差帧(即,相邻RGB帧之间的差异)作为替代的“轻量级”运动表示,其携带显著的运动信息并且在计算上高效。此外,我们开发了一个新的伪三维卷积模块,它将三维卷积分解为二维和一维卷积。所提出的模块利用特征空间中的残留信息来更好地结构化运动,并且配备有自我注意力机制,其有助于重新校准外观和运动特征。实验结果证实了残余帧以及所提出的伪3D卷积模块的效率和有效性。

英文摘要:

Human action recognition is regarded as a key cornerstone in domains such as surveillance or video understanding. Despite recent progress in the development of end-to-end solutions for video-based action recognition, achieving state-of-the-art performance still requires using auxiliary hand-crafted motion representations, e.g., optical flow, which are usually computationally demanding. In this work, we propose to use residual frames (i.e., differences between adjacent RGB frames) as an alternative "lightweight" motion representation, which carries salient motion information and is computationally efficient. In addition, we develop a new pseudo-3D convolution module which decouples 3D convolution into 2D and 1D convolution. The proposed module exploits residual information in the feature space to better structure motions, and is equipped with a self-attention mechanism that assists to recalibrate the appearance and motion features. Empirical results confirm the efficiency and effectiveness of residual frames as well as the proposed pseudo-3D convolution module.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑