SkelVIT：面向轻量级基于骨架的动作识别系统的视觉Transformer共识
SkelVIT: Consensus of Vision Transformers for a Lightweight Skeleton-Based Action Recognition System
AI总结：
本文提出SkelVit三层架构，通过生成多种伪图像表示、分别分类并共识融合，实现轻量级且鲁棒的基于骨架的动作识别，性能优于现有方法。
摘要：
基于骨架的动作识别因其对视角和光照变化具有鲁棒性，且处理效率远高于视频帧处理而受到许多研究者的关注。随着深度学习模型的出现，以伪图像形式表示骨架数据并应用CNN进行动作识别已变得非常流行。此后，研究集中在寻找形成伪图像的有效方法上。近年来，注意力网络，更具体地说是Transformer，在各种视觉问题中提供了有前景的结果。在本研究中，考察了ViT在基于骨架的动作识别中的有效性，并研究了其对伪图像表示方案的鲁棒性。为此，提出了一种三层架构SkelVit，该架构形成一组伪图像，对每个表示应用一个分类器，并组合它们的结果以找到最终的动作类别。通过一组实验对SkelVit的性能进行了全面检验。首先，通过将其与两种最先进的伪图像表示方法进行比较，研究了系统对表示的敏感性。然后，在两种实验设置中分别用CNN和ViT实现SkelVit的分类器，并比较它们的性能。在最后的实验设置中，通过应用具有不同数量分类器的模型来检验组合分类器的贡献。实验研究表明，所提出的系统凭借其轻量级表示方案取得了优于最先进方法的结果。还观察到，与CNN相比，视觉Transformer对初始伪图像表示的敏感性较低。尽管如此，即使使用视觉Transformer，通过分类器的共识也可以进一步提高识别性能。
英文摘要：
Skeleton-based action recognition receives the attention of many researchers as it is robust to viewpoint and illumination changes, and its processing is much more efficient than the processing of video frames. With the emergence of deep learning models, it has become very popular to represent the skeleton data in pseudo-image form and apply CNN for action recognition. Thereafter, studies concentrated on finding effective methods for forming pseudo-images. Recently, attention networks, more specifically transformers have provided promising results in various vision problems. In this study, the effectiveness of VIT for skeleton-based action recognition is examined and its robustness on the pseudo-image representation scheme is investigated. To this end, a three-level architecture, SkelVit is proposed, which forms a set of pseudo images, applies a classifier on each of the representations, and combines their results to find the final action class. The performance of SkelVit is examined thoroughly via a set of experiments. First, the sensitivity of the system to representation is investigated by comparing it with two of the state-of-the-art pseudo-image representation methods. Then, the classifiers of SkelVit are realized in two experimental setups by CNNs and VITs, and their performances are compared. In the final experimental setup, the contribution of combining classifiers is examined by applying the model with a different number of classifiers. Experimental studies reveal that the proposed system with its lightweight representation scheme achieves better results than the state-of-the-art methods. It is also observed that the vision transformer is less sensitive to the initial pseudo-image representation compared to CNN. Nevertheless, even with the vision transformer, the recognition performance can be further improved by the consensus of classifiers.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
