将符号表示为符号:一次性孤立手语识别以促进功能性手语技术
Representing Signs as Signs: One-Shot ISLR to Facilitate Functional Sign Language Technologies
AI总结:
提出一种跨语言的一次性孤立手语识别方法,通过预训练嵌入和密集向量搜索,在大型异语言词典上达到50.8% MRR,与DHH社区共创,推动可扩展手语技术。
摘要:
孤立手语识别(ISLR)对于可扩展的手语技术至关重要,然而特定语言的方法限制了当前模型。为解决这一问题,我们提出了一种一次性学习方法,该方法能够跨语言和不断演变的词汇进行泛化。我们的方法包括预训练一个模型,基于基本特征对手语进行嵌入,并使用密集向量搜索来快速、准确地识别未见过的符号。我们取得了最先进的结果,包括在一个包含10,235个来自训练集不同语言的独特符号的大型词典上实现了50.8%的一次性MRR。我们的方法在语言和支持集方面具有鲁棒性,为ISLR提供了一种可扩展、自适应的解决方案。该方法与聋人和听力障碍(DHH)社区共同创建,符合现实世界的需求,并推动了可扩展手语识别的发展。
英文摘要:
Isolated Sign Language Recognition (ISLR) is crucial for scalable sign language technology, yet language-specific approaches limit current models. To address this, we propose a one-shot learning approach that generalises across languages and evolving vocabularies. Our method involves pretraining a model to embed signs based on essential features and using a dense vector search for rapid, accurate recognition of unseen signs. We achieve state-of-the-art results, including 50.8% one-shot MRR on a large dictionary containing 10,235 unique signs from a different language than the training set. Our approach is robust across languages and support sets, offering a scalable, adaptable solution for ISLR. Co-created with the Deaf and Hard of Hearing (DHH) community, this method aligns with real-world needs, and advances scalable sign language recognition.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。