arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.07244cs.CV
收藏

基于深度学习与低成本手工设计特征的手语识别

Sign language recognition based on deep learning and low-cost handcrafted descriptors

São Paulo State University(圣保罗州立大学)

Alvaro Leandro Cavalcante Carneiro, Denis Henrique Pinheiro Salvadeo, Lucas de Brito Silva

AI总结:

针对现有手语识别系统计算成本高、落地难的问题,本研究结合低成本传感器与深度学习,提出基于边界框质心空间信息的手工特征方法,在AUTSL数据集上准确率提升7.96%,兼顾了准确率与计算效率。

摘要:

近年来,深度学习技术被用于开发生语识别系统,有望成为全球数百万听障人士的沟通工具。然而,构建这类系统存在固有挑战。首先,必须在手势执行中考虑尽可能多的语言学参数,以避免词语之间的歧义。此外,为推动所开发方案的实际落地,需确保所选技术具备现实可行性,避免使用昂贵、侵入性强或移动性差的传感器,同时规避计算要求极高的复杂深度学习架构。基于此,本研究旨在提出一种利用低成本传感器和技术的高效手语识别系统。为此,我们专门训练了一个目标检测模型,用于检测手语翻译者的脸部和手部,确保聚焦图像中最相关的区域,为分类器生成语义价值更高的输入。此外,我们提出了一种新颖的方法,利用边界框质心位置得到的空间信息来获取表征手部位置与运动的特征,从而提升手语的区分度。结果表明,我们的手工设计特征具备高效性:在AUTSL数据集上准确率提升了7.96%,同时新增参数不足70万个,额外推理时间不到10毫秒。这些发现凸显了我们的技术在计算成本与准确率之间取得良好平衡的潜力,使其成为实际手语识别应用中一种很有前景的方法。

英文摘要:

In recent years, deep learning techniques have been used to develop sign language recognition systems, potentially serving as a communication tool for millions of hearing-impaired individuals worldwide. However, there are inherent challenges in creating such systems. Firstly, it is important to consider as many linguistic parameters as possible in gesture execution to avoid ambiguity between words. Moreover, to facilitate the real-world adoption of the created solution, it is essential to ensure that the chosen technology is realistic, avoiding expensive, intrusive, or low-mobility sensors, as well as very complex deep learning architectures that impose high computational requirements. Based on this, our work aims to propose an efficient sign language recognition system that utilizes low-cost sensors and techniques. To this end, an object detection model was trained specifically for detecting the interpreter's face and hands, ensuring focus on the most relevant regions of the image and generating inputs with higher semantic value for the classifier. Additionally, we introduced a novel approach to obtain features representing hand location and movement by leveraging spatial information derived from centroid positions of bounding boxes, thereby enhancing sign discrimination. The results demonstrate the efficiency of our handcrafted features, increasing accuracy by 7.96% on the AUTSL dataset, while adding fewer than 700 thousand parameters and incurring less than 10 milliseconds of additional inference time. These findings highlight the potential of our technique to strike a favorable balance between computational cost and accuracy, making it a promising approach for practical sign language recognition applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑