DeepASL：实现无处不在的非侵入式单词和句子级手语翻译
DeepASL: Enabling Ubiquitous and Non-Intrusive Word and Sentence-Level Sign Language Translation
摘要：
聋人和听力正常的人之间存在着不可否认的沟通障碍。尽管手语翻译技术的创新旨在消除这种沟通障碍，但大多数现有的手语翻译系统要么是侵入性的，要么受到分辨率或环境照明条件的限制。此外，这些现有的系统只能执行单符号ASL翻译，而不是会话级翻译，这使得它们在日常通信场景中的用处要小得多。在这项工作中，我们通过介绍DeepASL填补了这一关键空白，DeepASL是一种基于深度学习的变革性手语翻译技术，可以在单词和句子层面实现无处不在的非侵入式美国手语（ASL）翻译。DeepASL使用红外光作为其传感机制，以非侵入性方式捕获ASL标志。它结合了一种新的分层双向深度递归神经网络（HB-RNN）和一个基于连接主义时间分类（CTC）的概率框架，分别用于单词级和句子级ASL翻译。为了评估其性能，我们从11名参与者中收集了7，306个样本，涵盖了56个常用的美国手语单词和100个美国手语句子。DeepASL在翻译看不见的ASL句子时达到了平均94.5%的单词级翻译准确率和平均8.2%的单词出错率。鉴于其良好的表现，我们相信DeepASL代表着打破聋人与听力正常的大多数人之间沟通障碍的重要一步，因此具有从根本上改变聋人生活的巨大潜力。
英文摘要：
There is an undeniable communication barrier between deaf people and people with normal hearing ability. Although innovations in sign language translation technology aim to tear down this communication barrier, the majority of existing sign language translation systems are either intrusive or constrained by resolution or ambient lighting conditions. Moreover, these existing systems can only perform single-sign ASL translation rather than sentence-level translation, making them much less useful in daily-life communication scenarios. In this work, we fill this critical gap by presenting DeepASL, a transformative deep learning-based sign language translation technology that enables ubiquitous and non-intrusive American Sign Language (ASL) translation at both word and sentence levels. DeepASL uses infrared light as its sensing mechanism to non-intrusively capture the ASL signs. It incorporates a novel hierarchical bidirectional deep recurrent neural network (HB-RNN) and a probabilistic framework based on Connectionist Temporal Classification (CTC) for word-level and sentence-level ASL translation respectively. To evaluate its performance, we have collected 7,306 samples from 11 participants, covering 56 commonly used ASL words and 100 ASL sentences. DeepASL achieves an average 94.5% word-level translation accuracy and an average 8.2% word error rate on translating unseen ASL sentences. Given its promising performance, we believe DeepASL represents a significant step towards breaking the communication barrier between deaf people and hearing majority, and thus has the significant potential to fundamentally change deaf people's lives.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
