通过少样本原型网络实现高效的手语识别
Data-Efficient American Sign Language Recognition via Few-Shot Prototypical Networks
AI总结：
本文提出一种少样本原型网络框架，通过时空图卷积网络和多尺度时间聚合模块提升手语识别的准确率与泛化能力。
摘要：
孤立手语识别（ISLR）对于弥合聋人和听力受损者与听力世界之间的沟通差距至关重要。然而，稳健的ISLR从根本上受到数据稀缺和手语词汇长尾分布的限制，其中为数千种独特手语收集足够的示例成本高昂。标准分类方法在这些条件下表现不佳，往往过度拟合常见类别而无法泛化到罕见类别。为了解决这一瓶颈，我们提出了一种适应于基于骨架的编码器的少样本原型网络框架。不同于传统分类器学习固定决策边界，我们的方法利用经验训练来学习一个语义度量空间，其中手语根据其与动态类别原型的接近程度进行分类。我们整合了一个时空图卷积网络（ST-GCN）与一个新颖的多尺度时间聚合（MSTA）模块，以捕捉快速和流畅的运动动态。在WLASL数据集上的实验结果证明了这种度量学习范式的优越性：我们的模型在测试集上实现了43.75%的Top-1准确率和77.10%的Top-5准确率。关键的是，这比具有相同骨干架构的标准分类基线高出超过13%，证明了原型训练策略在标准分类失败的数据稀缺情况下表现更优。此外，该模型表现出强大的零样本泛化能力，在未见过的SignASL数据集上无需微调即可达到近30%的准确率，为在有限数据下识别大量手语词汇提供了可扩展的路径。
英文摘要：
Isolated Sign Language Recognition (ISLR) is critical for bridging the communication gap between the Deaf and Hard-of-Hearing (DHH) community and the hearing world. However, robust ISLR is fundamentally constrained by data scarcity and the long-tail distribution of sign vocabulary, where gathering sufficient examples for thousands of unique signs is prohibitively expensive. Standard classification approaches struggle under these conditions, often overfitting to frequent classes while failing to generalize to rare ones. To address this bottleneck, we propose a Few-Shot Prototypical Network framework adapted for a skeleton based encoder. Unlike traditional classifiers that learn fixed decision boundaries, our approach utilizes episodic training to learn a semantic metric space where signs are classified based on their proximity to dynamic class prototypes. We integrate a Spatiotemporal Graph Convolutional Network (ST-GCN) with a novel Multi-Scale Temporal Aggregation (MSTA) module to capture both rapid and fluid motion dynamics. Experimental results on the WLASL dataset demonstrate the superiority of this metric learning paradigm: our model achieves 43.75% Top-1 and 77.10% Top-5 accuracy on the test set. Crucially, this outperforms a standard classification baseline sharing the identical backbone architecture by over 13%, proving that the prototypical training strategy effectively outperforms in a data scarce situation where standard classification fails. Furthermore, the model exhibits strong zero-shot generalization, achieving nearly 30% accuracy on the unseen SignASL dataset without fine-tuning, offering a scalable pathway for recognizing extensive sign vocabularies with limited data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
