UniCon-Former：统一卷积Transformer是手势识别的全部所需

中文摘要 AI 辅助

卷积神经网络（CNNs）能高效捕获局部特征，但因感受野有限难以捕捉全局上下文；Transformer则通过自注意力有效捕获全局依赖，但存在高冗余和计算成本高的问题。因此，为利用CNNs和Transformer的优势，本文提出统一模型UniCon-Former，旨在在动态手势识别任务上提供鲁棒且高效的性能。该统一方法助力模型同时学习局部与全局特征：在每个Transformer阶段开始时，卷积投影可降低Transformer模块输入向量的维度，在各Transformer阶段形成金字塔结构。这些特性使UniCon-Former相比普通Transformer减少资源使用，具备学习多尺度和高分辨率特征的灵活性，而这正是手势识别所需的。本文在NVGesture和Briareo数据集上开展实验，以更少的参数和MACs（乘法累加运算）取得了当前最优（SOTA）结果。

英文摘要

Convolutional Neural Networks (CNNs) capture local features efficiently but struggle with global context due to their limited receptive field. On the other hand, transformers effectively capture global dependencies through self-attention but suffer from high redundancy and computational costs. Thus, to leverage the advantages of both CNNs and transformers, we propose a unified model (UniCon-Former) that aims to provide robust and efficient performance on dynamic hand gesture recognition. The unified approach helps the model to learn both local and global features. At the beginning of each transformer stage, the convolution projections help in decreasing the dimension of the input vectors of the transformer block. This creates a pyramidal structure at each transformer stage. These features enable the UniCon-Former to reduce resource usage than vanilla transformers, making it flexible for learning multi-scale and high-resolution features, which is required in hand gesture recognition. We have performed experiments with NVGesture and Briareo datasets and achieved state-of-the-art results with fewer parameters and MACs.

发表机构

• Indian Institute of Technology Kharagpur（印度克勒格布尔印度理工学院）
• Indian Institute of Technology Roorkee（印度鲁基印度理工学院）

机构由 AI 辅助整理，请以论文原文为准。