arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.12967cs.CV
收藏

基于时间通道拓扑增强的神经网络用于基于小波变换的动作识别

Temporal-Channel Topology Enhanced Network for Skeleton-Based Action Recognition

National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所模式识别国家重点实验室);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)

Jinzhao Luo, Lu Zhou, Guibo Zhu, Guojing Ge, Beiying Yang, Jinqiao Wang

摘要:

基于手势的动作识别由于其高效性和鲁棒性而在近年来变得流行。现有的拓扑建模方法大多采用图卷积网络(GCN),但基于GCN的拓扑建模方法在长距离相关建模和泛化能力方面存在局限性。相比之下,卷积神经网络(CNN)用于拓扑建模的潜力尚未得到充分开发。在本文中,我们提出了一种新的CNN架构,时间通道拓扑增强网络(TCTE-Net),学习空间和时间拓扑结构的基于神经网络的动作识别。TCTE-Net由两个模块组成:时间-通道焦点模块,它学习时间-通道焦点矩阵以识别最关键的特征表示,以及动态通道拓扑注意模块,它动态学习空间拓扑特征,并将它们与注意力机制融合以模拟长距离通道拓扑。我们在NTU RGB+D,NTU RGB+D 120和FineGym数据集上进行实验。与基于CNN的方法相比,TCTE-Net显示出最先进的性能,并且与基于GCN的方法相比,具有更优越的性能。该代码可在https://github.com/aikuniverse/TCTE-Net上获得。

英文摘要:

Skeleton-based action recognition has become popular in recent years due to its efficiency and robustness. Most current methods adopt graph convolutional network (GCN) for topology modeling, but GCN-based methods are limited in long-distance correlation modeling and generalizability. In contrast, the potential of convolutional neural network (CNN) for topology modeling has not been fully explored. In this paper, we propose a novel CNN architecture, Temporal-Channel Topology Enhanced Network (TCTE-Net), to learn spatial and temporal topologies for skeleton-based action recognition. The TCTE-Net consists of two modules: the Temporal-Channel Focus module, which learns a temporal-channel focus matrix to identify the most critical feature representations, and the Dynamic Channel Topology Attention module, which dynamically learns spatial topological features, and fuses them with an attention mechanism to model long-distance channel-wise topology. We conduct experiments on NTU RGB+D, NTU RGB+D 120, and FineGym datasets. TCTE-Net shows state-of-the-art performance compared to CNN-based methods and achieves superior performance compared to GCN-based methods. The code is available at https://github.com/aikuniverse/TCTE-Net.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑