儿童自动语音识别(ASR)错误的因果分析:量化生理、认知和外在因素的影响
Causal Analysis of ASR Errors for Children: Quantifying the Impact of Physiological, Cognitive, and Extrinsic Factors
AI总结:
针对儿童自动语音识别(ASR)模型词错误率(WER)高于成人的问题,本研究通过因果分析量化生理、认知和外在因素的影响,发现年龄和词数影响最大,微调可降低对生理和认知因素的敏感性。
摘要:
近年来,儿童自动语音识别(ASR)系统的使用日益增加,推动了旨在提高儿童语音模型准确性的研究工作。当前方法要么直接使用开源语音基础模型(SFM),要么用儿童语音数据对其进行微调。这些SFM无论是否开源或针对儿童微调,与成人语音相比往往表现出更高的词错误率(WER)。然而,目前缺乏对SFM性能下降原因的系统性分析。理解并解决这种性能差异背后的原因,对于提高儿童语音SFM的准确性至关重要。本研究通过调查儿童语音中准确性下降的原因和WER的主要贡献因素来填补这一空白。在研究的第一部分,我们在两个儿童语音语料库上对两种自监督SFM(Wav2Vec2.0和Hubert)和两种弱监督SFM(Whisper和MMS)进行了跨年龄组的综合基准研究,为第二部分的因果推理分析建立了原始数据。在研究的第二部分,我们使用因果推理分析了生理因素(年龄、性别)、认知因素(发音能力)和外在因素(词汇难度、背景噪音和词数)对儿童语音SFM准确性的影响。结果表明,生理因素(年龄)和特定外在因素(音频中的词数)对准确性的影响最大,其次是背景噪音和发音能力。在儿童语音上微调SFM降低了对生理和认知因素的敏感性,而对音频中词数的敏感性仍然存在。
英文摘要:
The increasing use of children's automatic speech recognition (ASR) systems has spurred research efforts to improve the accuracy of models designed for children's speech in recent years. The current approach utilizes either open-source speech foundation models (SFMs) directly or fine-tuning them with children's speech data. These SFMs, whether open-source or fine-tuned for children, often exhibit higher word error rates (WERs) compared to adult speech. However, there is a lack of systemic analysis of the cause of this degraded performance of SFMs. Understanding and addressing the reasons behind this performance disparity is crucial for improving the accuracy of SFMs for children's speech. Our study addresses this gap by investigating the causes of accuracy degradation and the primary contributors to WER in children's speech. In the first part of the study, we conduct a comprehensive benchmarking study on two self-supervised SFMs (Wav2Vec2.0 and Hubert) and two weakly supervised SFMs (Whisper and MMS) across various age groups on two children speech corpora, establishing the raw data for the causal inference analysis in the second part. In the second part of the study, we analyze the impact of physiological factors (age, gender), cognitive factors (pronunciation ability), and external factors (vocabulary difficulty, background noise, and word count) on SFM accuracy in children's speech using causal inference. The results indicate that physiology (age) and particular external factor (number of words in audio) have the highest impact on accuracy, followed by background noise and pronunciation ability. Fine-tuning SFMs on children's speech reduces sensitivity to physiological and cognitive factors, while sensitivity to the number of words in audio persists. Keywords: Children's ASR, Speech Foundational Models, Causal Inference, Physiology, Cognition, Pronunciation
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。