arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2106.08313eess.AS
收藏

构音障碍言语的口语理解预训练策略研究

A Study into Pre-training Strategies for Spoken Language Understanding on Dysarthric Speech

Pu Wang, Bagher BabaAli, Hugo Van hamme

摘要:

端到端(E2E)口语理解(SLU)系统通过将语音直接映射到具有槽值的意图来避免中间文本表示。这种方法需要大量的特定领域的训练数据。在低资源情况下,这是一个主要问题,例如,在本研究中处理SLU构音障碍的讲话。针对自动语音识别目标的SLU模型的预训练部分有所帮助,但没有研究表明,SLU对构音障碍语音的益处在多大程度上来自于从其他构音障碍语音任务转移的知识。本文探讨了构音障碍言语的SLU任务预训练策略的效率。设计的SLU系统包括一个TDNN声学模型的特征编码和一个胶囊网络的意图和时隙解码。声学模型的预训练分为两个阶段:使用正常语音语料库进行初始化,以及对构音障碍和正常语音的混合进行微调。本文通过引入可懂度分数作为构音障碍语音损伤程度的度量指标,定量分析构音障碍语音的泛化程度与病理严重程度的关系。

英文摘要:

End-to-end (E2E) spoken language understanding (SLU) systems avoid an intermediate textual representation by mapping speech directly into intents with slot values. This approach requires considerable domain-specific training data. In low-resource scenarios this is a major concern, e.g., in the present study dealing with SLU for dysarthric speech. Pretraining part of the SLU model for automatic speech recognition targets helps but no research has shown to which extent SLU on dysarthric speech benefits from knowledge transferred from other dysarthric speech tasks. This paper investigates the efficiency of pre-training strategies for SLU tasks on dysarthric speech. The designed SLU system consists of a TDNN acoustic model for feature encoding and a capsule network for intent and slot decoding. The acoustic model is pre-trained in two stages: initialization with a corpus of normal speech and finetuning on a mixture of dysarthric and normal speech. By introducing the intelligibility score as a metric of the impairment severity, this paper quantitatively analyzes the relation between generalization and pathology severity for dysarthric speech.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑