arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2106.01451cs.CLcs.AI
收藏

语音识别中基于注意力的上下文语言模型自适应

Attention-based Contextual Language Model Adaptation for Speech Recognition

Amazon Alexa(亚马逊Alexa)

Richard Diehl Martinez, Scott Novotney, Ivan Bulyko, Ariya Rastrow, Andreas Stolcke, Ankur Gandhe

摘要:

用于自动语音识别(ASR)的语言建模(LM)通常不包括话语级上下文信息。然而,对于像语音助理这样的一些领域,附加的上下文(诸如说出话语的时间)提供了丰富的输入信号。我们引入了一种注意力机制,用于在文本和非语言上下文数据上训练神经语音识别语言模型。当应用于一个流行的语音助手平台收集的大量去识别的话语数据集时,我们的方法相对于不包含上下文信息的标准LM减少了7.0%的困惑。当从数据集的长尾中提取的话语进行评估时,我们的方法相对于标准LM提高了9.0%的困惑度,与最先进的上下文LM模型相比,相对提高了2.8%。

英文摘要:

Language modeling (LM) for automatic speech recognition (ASR) does not usually incorporate utterance level contextual information. For some domains like voice assistants, however, additional context, such as the time at which an utterance was spoken, provides a rich input signal. We introduce an attention mechanism for training neural speech recognition language models on both text and non-linguistic contextual data. When applied to a large de-identified dataset of utterances collected by a popular voice assistant platform, our method reduces perplexity by 7.0% relative over a standard LM that does not incorporate contextual information. When evaluated on utterances extracted from the long tail of the dataset, our method improves perplexity by 9.0% relative over a standard LM and by over 2.8% relative when compared to a state-of-the-art model for contextual LM.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑