面向LVCSR的未来向量增强PDKEEPAAA语言模型
Future Vector Enhanced LSTM Language Model for LVCSR
摘要：
语言模型在大词汇量连续语音识别中起着重要的作用。然而，传统的语言模型只能预测下一个单一的单词与给定的历史，而连续的预测序列的单词通常是需要和有用的LVCSR。训练中的单字预测建模与读取需求中的长期序列预测之间的不匹配可能导致性能下降。本文提出了一种新的使用未来向量的增强型长短期记忆（LSTM）LM。除了给定的历史，序列的其余部分也将被未来的向量嵌入。这个未来的向量可以与LSTM LM合并，因此它能够建模更长期的序列级信息。实验结果表明，提出的新LSTM LM在长期序列预测的BLEU分数上获得了更好的结果。对于语音识别重评分，尽管所提出的LSTM LM获得了非常小的增益，但新模型似乎与传统的LSTM LM获得了很大的互补性。使用新的和传统的LSTM LM进行重新评分可以在字错误率上实现非常大的改善。
英文摘要：
Language models (LM) play an important role in large vocabulary continuous speech recognition (LVCSR). However, traditional language models only predict next single word with given history, while the consecutive predictions on a sequence of words are usually demanded and useful in LVCSR. The mismatch between the single word prediction modeling in trained and the long term sequence prediction in read demands may lead to the performance degradation. In this paper, a novel enhanced long short-term memory (LSTM) LM using the future vector is proposed. In addition to the given history, the rest of the sequence will be also embedded by future vectors. This future vector can be incorporated with the LSTM LM, so it has the ability to model much longer term sequence level information. Experiments show that, the proposed new LSTM LM gets a better result on BLEU scores for long term sequence prediction. For the speech recognition rescoring, although the proposed LSTM LM obtains very slight gains, the new model seems obtain the great complementary with the conventional LSTM LM. Rescoring using both the new and conventional LSTM LMs can achieve a very large improvement on the word error rate.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
