arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1606.07470cs.CLstat.ML
收藏

NN-grams:统一神经网络和n-gram语言模型用于语音识别

NN-grams: Unifying neural network and n-gram language models for Speech Recognition

Google Inc.(谷歌公司)

Babak Damavandi, Shankar Kumar, Noam Shazeer, Antoine Bruguier

摘要:

我们提出了NN-元,一种新的,混合的语言模型集成了n-gram和神经网络(NN)的语音识别。该模型将单词历史和n-gram计数作为输入。因此,它结合了n-gram模型的记忆能力和可扩展性与神经网络的泛化能力。我们报告了在26 B单词上训练模型的实验。NN元语法在运行时是高效的,因为它们不包括输出soft-max层。该模型使用噪声对比估计(NCE)进行训练,该方法将神经网络的估计问题转化为数据样本和噪声样本之间的二元分类问题。我们目前的结果与噪声样本来自一个n-gram分布或从语音识别晶格。在意大利语语音识别听写任务中,NN-gram的性能优于n-gram模型。

英文摘要:

We present NN-grams, a novel, hybrid language model integrating n-grams and neural networks (NN) for speech recognition. The model takes as input both word histories as well as n-gram counts. Thus, it combines the memorization capacity and scalability of an n-gram model with the generalization ability of neural networks. We report experiments where the model is trained on 26B words. NN-grams are efficient at run-time since they do not include an output soft-max layer. The model is trained using noise contrastive estimation (NCE), an approach that transforms the estimation problem of neural networks into one of binary classification between data samples and noise samples. We present results with noise samples derived from either an n-gram distribution or from speech recognition lattices. NN-grams outperforms an n-gram model on an Italian speech recognition dictation task.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑