用于孤立印度手语识别的分层窗口图注意力网络与大规模数据集
Hierarchical Windowed Graph Attention Network and a Large Scale Dataset for Isolated Indian Sign Language Recognition
AI总结：
针对印度手语数据缺乏问题，本文构建了包含40033个视频的大规模孤立ISL数据集，并提出基于上半身骨架图的HWGAT模型，在多个手语数据集上显著提升了识别性能。
摘要：
自动手语（SL）识别是计算机视觉领域的一项重要任务。为了构建鲁棒的SL识别系统，我们需要大量数据，而这在印度手语（ISL）中尤为缺乏。本文引入了一个大规模的孤立ISL数据集，以及一种基于骨架图结构的新型SL识别模型。该数据集涵盖了聋哑人社区日常使用的2002个常用词汇，由20名（10名男性和10名女性）聋哑成人手语者录制（包含40033个视频）。我们利用人体上半身骨架图，提出了一种名为分层窗口图注意力网络（HWGAT）的SL识别模型。HWGAT通过关注人体骨架图所诱导的不同身体部位，试图捕捉独特的动作。我们通过大量实验评估了所提数据集的效用和模型的有效性。我们在所提出的数据集上对模型进行了预训练，并在不同的手语数据集上进行了微调，与现有的基于关键点的最先进模型相比，在INCLUDE、LSA64、AUTSL和WLASL上分别进一步提升了1.10、0.46、0.78和6.84个百分点的性能。
英文摘要：
Automatic Sign Language (SL) recognition is an important task in the computer vision community. To build a robust SL recognition system, we need a considerable amount of data which is lacking particularly in Indian sign language (ISL). In this paper, we introduce a large-scale isolated ISL dataset and a novel SL recognition model based on skeleton graph structure. The dataset covers 2002 daily used common words in the deaf community recorded by 20 (10 male and 10 female) deaf adult signers (contains 40033 videos). We propose a SL recognition model namely Hierarchical Windowed Graph Attention Network (HWGAT) by utilizing the human upper body skeleton graph. The HWGAT tries to capture distinctive motions by giving attention to different body parts induced by the human skeleton graph. The utility of the proposed dataset and the usefulness of our model are evaluated through extensive experiments. We pre-trained the proposed model on the presented dataset and fine-tuned it across different sign language datasets further boosting the performance of 1.10, 0.46, 0.78, and 6.84 percentage points on INCLUDE, LSA64, AUTSL and WLASL respectively compared to the existing state-of-the-art keypoints-based models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
