arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.19443cs.SDcs.AIeess.AS
收藏

探究自动语音识别系统中基于神经网络的声学模型所编码的信息

Probing the Information Encoded in Neural-based Acoustic Models of Automatic Speech Recognition Systems

Avignon University(阿维尼翁大学);Nantes University(南特大学)

Quentin Raymondaud, Mickael Rouvier, Richard Dufour

AI总结:

本文提出通过不同层中间表示在多类语音任务上评估ASR声学模型,以识别其编码信息的类型与位置,并揭示低层结构化、高层剔除无关信息的规律。

摘要:

深度学习架构已在许多研究领域的性能方面取得显著进展。因此,自动语音识别(ASR)领域受益于这些科技进步,尤其是在声学建模方面,如今已整合深度神经网络架构。然而,这些性能提升也使得通过这些黑盒架构学习并传递的信息在复杂性上有所增加。继大量关于神经网络可解释性的研究之后,本文提出一种协议,旨在确定ASR声学模型(AM)中包含哪些信息以及这些信息位于何处。为此,我们建议使用中间表示(此处指不同层级的表示)在一组既定任务上评估AM的性能。根据性能变化和目标任务,我们可以提出假设:在架构的不同阶段,哪些信息得到增强或受到干扰。实验在说话人验证、声学环境分类、性别分类、速度失真检测系统以及语音情感/情绪识别上进行。分析表明,基于神经网络的AM包含异质信息,而这些信息似乎出人意料地与音素识别无关,例如情感、情绪或说话人身份。总体而言,低层隐藏层似乎有助于信息的结构化,而高层隐藏层则倾向于删除对音素识别无用的信息。

英文摘要:

Deep learning architectures have made significant progress in terms of performance in many research areas. The automatic speech recognition (ASR) field has thus benefited from these scientific and technological advances, particularly for acoustic modeling, now integrating deep neural network architectures. However, these performance gains have translated into increased complexity regarding the information learned and conveyed through these black-box architectures. Following many researches in neural networks interpretability, we propose in this article a protocol that aims to determine which and where information is located in an ASR acoustic model (AM). To do so, we propose to evaluate AM performance on a determined set of tasks using intermediate representations (here, at different layer levels). Regarding the performance variation and targeted tasks, we can emit hypothesis about which information is enhanced or perturbed at different architecture steps. Experiments are performed on both speaker verification, acoustic environment classification, gender classification, tempo-distortion detection systems and speech sentiment/emotion identification. Analysis showed that neural-based AMs hold heterogeneous information that seems surprisingly uncorrelated with phoneme recognition, such as emotion, sentiment or speaker identity. The low-level hidden layers globally appears useful for the structuring of information while the upper ones would tend to delete useless information for phoneme recognition.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑