在大语言模型时代的手语识别
Sign Language Recognition in the Age of LLMs
AI总结：
研究探讨现代VLMs在零样本条件下进行孤立手语识别的能力，发现开源模型表现欠佳，而大模型在精度上表现突出，强调模型规模和训练数据多样性的重要性。
摘要：
近期视觉语言模型（VLMs）在多种多模态推理任务中表现出色。这引发了一个问题：此类通用模型能否在无需任务特定训练的情况下解决专门的视觉识别问题，如孤立手语识别（ISLR）？本文研究了现代VLMs在零样本设置下执行ISLR的能力。我们在WLASL300基准上评估了多个开源和专有VLMs。实验表明，在仅提示的零样本推理下，当前开源VLMs在精度上远逊于经典监督ISLR分类器。然而，后续实验表明这些模型能捕捉到手语与文本描述之间的部分视觉-语义对齐。较大的专有模型实现了显著更高的精度，突显了模型规模和训练数据多样性的重要性。所有代码均在GitHub上公开。
英文摘要：
Recent Vision Language Models (VLMs) have demonstrated strong performance across a wide range of multimodal reasoning tasks. This raises the question of whether such general-purpose models can also address specialized visual recognition problems such as isolated sign language recognition (ISLR) without task-specific training. In this work, we investigate the capability of modern VLMs to perform ISLR in a zero-shot setting. We evaluate several open-source and proprietary VLMs on the WLASL300 benchmark. Our experiments show that, under prompt-only zero-shot inference, current open-source VLMs remain far behind classic supervised ISLR classifiers by a wide margin. However, follow-up experiments reveal that these models capture partial visual-semantic alignment between signs and text descriptions. Larger proprietary models achieve substantially higher accuracy, highlighting the importance of model scale and training data diversity. All our code is publicly available on GitHub.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
