语音-手势交互的图模型分析用于失语症严重程度估计
Graph Modelling Analysis of Speech-Gesture Interaction for Aphasia Severity Estimation
AI总结：
本文提出基于图神经网络的框架，通过分析语音和手势的交互来自动估计失语症严重程度。
摘要：
失语症是一种由大脑负责语言的区域损伤引起的获得性语言障碍。失语症可能损害书面和口语语言的使用和理解。西方失语症评估量表-修订版（WAB-R）是语言病理学家（SLPs）用来评估失语症类型和严重程度的评估工具。由于WAB-R测量孤立的言语技能，因此对话语生产进行评估作为日常语言能力的更全面表示引起了越来越多的兴趣。最近的语音分析进展集中在从自发语音中自动估计失语症严重程度上，主要依赖于孤立的言语或声学特征。在本工作中，我们提出了一种基于图神经网络的框架来估计失语症严重程度。我们将每个参与者的对话表示为有向多模态图，其中节点代表词汇项和手势，边编码词-词、手势-词和词-手势转换。使用GraphSAGE来学习参与者级别的嵌入，从而整合来自邻居和整体图结构的信息。我们的结果表明，失语症严重程度并不编码在孤立的词汇分布中，而是从语音和手势之间的结构互动中涌现出来。所提出的架构提供了一种可靠的自动失语症评估方法，可能用于床旁筛查和基于远程医疗的监测。
英文摘要：
Aphasia is an acquired language disorder caused by injury to the regions of the brain that are responsible for language. Aphasia may impair the use and comprehension of written and spoken language. The Western Aphasia Battery-Revised (WAB-R) is an assessment tool administered by speech-language pathologists (SLPs) to evaluate the aphasia type and severity. Because the WAB-R measures isolated linguistic skills, there has been growing interest in the assessment of discourse production as a more holistic representation of everyday language abilities. Recent advancements in speech analysis focus on automated estimation of aphasia severity from spontaneous speech, relying mostly in isolated linguistic or acoustical features. In this work, we propose a graph neural network-based framework for estimating aphasia severity. We represented each participant's discourse as a directed multi-modal graph, where nodes represent lexical items and gestures and edges encode word-word, gesture-word, and word-gesture transitions. GraphSAGE is employed to learn participant-level embeddings, thus integrating information from immediate neighbors and overall graph structure. Our results suggest that aphasia severity is not encoded in isolated lexical distribution, but rather emerges from structured interactions between speech and gesture. The proposed architecture offers a reliable automated aphasia assessment, with possible uses in bedside screening and telehealth-based monitoring.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
