arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2107.11818cs.CVcs.AI
收藏

基于级联BdSL网络的孟加拉语手语识别

Bangla sign language recognition using concatenated BdSL network

Islamic University of Technology (IUT)(伊斯兰科技大学)

Thasin Abedin, Khondokar S. S. Prottoy, Ayana Moshruba, Safayat Bin Hakim

摘要:

手语是听障人士和聋哑人士唯一的沟通媒介。因此,与普通大众的沟通始终是这一少数群体面临的挑战。特别是在孟加拉手语(BdSL)中,有38个字母,其中一些符号几乎相同。因此,在BdSL识别中,除了从传统卷积神经网络(CNN)提取的视觉特征之外,手的姿势是一个重要因素。本文提出了一种新的结构“级联BdSL网络”,它由一个基于CNN的图像网络和一个姿态估计网络组成。在图像网络获取视觉特征的同时,通过姿态估计网络获取手部关键点的相对位置,以获得额外的特征来处理BdSL符号的复杂性。实验结果表明,该方法在测试集上获得了91.51%的正确率,证明了该方法的有效性。

英文摘要:

Sign language is the only medium of communication for the hearing impaired and the deaf and dumb community. Communication with the general mass is thus always a challenge for this minority group. Especially in Bangla sign language (BdSL), there are 38 alphabets with some having nearly identical symbols. As a result, in BdSL recognition, the posture of hand is an important factor in addition to visual features extracted from traditional Convolutional Neural Network (CNN). In this paper, a novel architecture "Concatenated BdSL Network" is proposed which consists of a CNN based image network and a pose estimation network. While the image network gets the visual features, the relative positions of hand keypoints are taken by the pose estimation network to obtain the additional features to deal with the complexity of the BdSL symbols. A score of 91.51% was achieved by this novel approach in test set and the effectiveness of the additional pose estimation network is suggested by the experimental results.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑