arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.20171cs.CLcs.CV
收藏

将符号表示为符号:一次性孤立手语识别以促进功能性手语技术

Representing Signs as Signs: One-Shot ISLR to Facilitate Functional Sign Language Technologies

Ghent University(根特大学);imec(imec研究院);IDLab-AIRO(IDLab-AIRO实验室)

Toon Vandendriessche, Mathieu De Coster, Annelies Lejon, Joni Dambre

AI总结:

提出一种跨语言的一次性孤立手语识别方法,通过预训练嵌入和密集向量搜索,在大型异语言词典上达到50.8% MRR,与DHH社区共创,推动可扩展手语技术。

摘要:

孤立手语识别(ISLR)对于可扩展的手语技术至关重要,然而特定语言的方法限制了当前模型。为解决这一问题,我们提出了一种一次性学习方法,该方法能够跨语言和不断演变的词汇进行泛化。我们的方法包括预训练一个模型,基于基本特征对手语进行嵌入,并使用密集向量搜索来快速、准确地识别未见过的符号。我们取得了最先进的结果,包括在一个包含10,235个来自训练集不同语言的独特符号的大型词典上实现了50.8%的一次性MRR。我们的方法在语言和支持集方面具有鲁棒性,为ISLR提供了一种可扩展、自适应的解决方案。该方法与聋人和听力障碍(DHH)社区共同创建,符合现实世界的需求,并推动了可扩展手语识别的发展。

英文摘要:

Isolated Sign Language Recognition (ISLR) is crucial for scalable sign language technology, yet language-specific approaches limit current models. To address this, we propose a one-shot learning approach that generalises across languages and evolving vocabularies. Our method involves pretraining a model to embed signs based on essential features and using a dense vector search for rapid, accurate recognition of unseen signs. We achieve state-of-the-art results, including 50.8% one-shot MRR on a large dictionary containing 10,235 unique signs from a different language than the training set. Our approach is robust across languages and support sets, offering a scalable, adaptable solution for ISLR. Co-created with the Deaf and Hard of Hearing (DHH) community, this method aligns with real-world needs, and advances scalable sign language recognition.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑