基于面部肌肉运动可解释归因向量的口吃言语不流利预测
Stuttering Speech Disfluency Prediction using Explainable Attribution Vectors of Facial Muscle Movements
摘要:
口吃等言语障碍通过无意识的重复、重复和阻塞声音和音节来破坏正常的言语流畅性。除了这些对语言流畅性的干扰外,大多数口吃的成年人(AWS)在口吃之前,期间和之后还经历了许多可观察到的次要行为,通常涉及面部肌肉。最近的研究已经探索了使用基于人工智能(AI)的算法从语音发声期间的呼吸率、音频等自动检测口吃。然而,大多数方法需要受控的环境和/或侵入性可穿戴传感器,并且无法解释为什么做出决定(流利vs口吃)。我们假设AWS中的语音前面部活动可以非侵入性地捕获,包含足够的信息来准确地将即将到来的话语分类为流利或口吃。为此,本文提出了一种新的可解释AI(XAI)辅助卷积神经网络(CNN)分类器,通过学习AWS的时间面部肌肉运动模式来预测不久的将来的口吃,并解释了所涉及的重要面部肌肉和动作。统计分析显示,颊肌(p<0.005)和唇肌(p<0.005)预测口吃的患病率显着较高,并显示出有助于唤醒和期待说话的行为。对这些上面部肌肉和下面部肌肉的时间研究可以促进口吃的早期检测,促进口吃的自动评估,并通过提供实时的自动非侵入性反馈而在行为治疗中具有应用。
英文摘要:
Speech disorders such as stuttering disrupt the normal fluency of speech by involuntary repetitions, prolongations and blocking of sounds and syllables. In addition to these disruptions to speech fluency, most adults who stutter (AWS) also experience numerous observable secondary behaviors before, during, and after a stuttering moment, often involving the facial muscles. Recent studies have explored automatic detection of stuttering using Artificial Intelligence (AI) based algorithm from respiratory rate, audio, etc. during speech utterance. However, most methods require controlled environments and/or invasive wearable sensors, and are unable explain why a decision (fluent vs stuttered) was made. We hypothesize that pre-speech facial activity in AWS, which can be captured non-invasively, contains enough information to accurately classify the upcoming utterance as either fluent or stuttered. Towards this end, this paper proposes a novel explainable AI (XAI) assisted convolutional neural network (CNN) classifier to predict near future stuttering by learning temporal facial muscle movement patterns of AWS and explains the important facial muscles and actions involved. Statistical analyses reveal significantly high prevalence of cheek muscles (p<0.005) and lip muscles (p<0.005) to predict stuttering and shows a behavior conducive of arousal and anticipation to speak. The temporal study of these upper and lower facial muscles may facilitate early detection of stuttering, promote automated assessment of stuttering and have application in behavioral therapies by providing automatic non-invasive feedback in realtime.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。