一种基于联合频谱-时间关系思维的声学建模框架
A Joint Spectro-Temporal Relational Thinking Based Acoustic Modeling Framework
AI总结：
本文提出一种基于联合频谱-时间关系思维的声学建模框架，通过生成概率图建模跨时频域的语音片段关系并聚合嵌入到潜在表示中，在TIMIT数据集上音素识别任务提升7.82%，显著改善元音识别能力。
摘要：
关系思维是指人类形成关于感觉信号与先验知识之间关系的心理印象，并将其纳入其世界模型的固有能力。尽管关系思维在人类理解语音中起着至关重要的作用，但它尚未被任何人工语音识别系统所利用。最近有一些尝试来纠正这一疏忽，但这些尝试仅限于仅在时域中操作的粗略话语级模型。为了缩小人工系统与人类能力之间的差距，本文提出了一种新颖的基于频谱-时间关系思维的声学建模框架。具体而言，它首先生成大量概率图来建模跨时间和频率域的语音片段之间的关系。然后，根植于这些图中每对节点的关系信息被聚合并嵌入到可被下游任务利用的潜在表示中。基于此框架构建的模型在TIMIT数据集上的音素识别任务中取得了7.82%的提升，优于现有最先进的系统。深入分析进一步揭示，我们提出的关系思维建模主要提升了模型识别元音的能力，这些元音最容易被音素识别器混淆。
英文摘要：
Relational thinking refers to the inherent ability of humans to form mental impressions about relations between sensory signals and prior knowledge, and subsequently incorporate them into their model of their world. Despite the crucial role relational thinking plays in human understanding of speech, it has yet to be leveraged in any artificial speech recognition systems. Recently, there have been some attempts to correct this oversight, but these have been limited to coarse utterance-level models that operate exclusively in the time domain. In an attempt to narrow the gap between artificial systems and human abilities, this paper presents a novel spectro-temporal relational thinking based acoustic modeling framework. Specifically, it first generates numerous probabilistic graphs to model the relationships among speech segments across both time and frequency domains. The relational information rooted in every pair of nodes within these graphs is then aggregated and embedded into latent representations that can be utilized by downstream tasks. Models built upon this framework outperform state-of-the-art systems with a 7.82\% improvement in phoneme recognition tasks over the TIMIT dataset. In-depth analyses further reveal that our proposed relational thinking modeling mainly improves the model's ability to recognize vowels, which are the most likely to be confused by phoneme recognizers.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
