arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1602.06064cs.CL
收藏

带噪声对比估计的双向神经网络语言模型训练

On Training Bi-directional Neural Network Language Model with Noise Contrastive Estimation

Shanghai Jiao Tong University(上海交通大学);MIT(麻省理工学院);Microsoft Research(微软研究院)

Tianxing He, Yu Zhang, Jasha Droppo, Kai Yu

摘要:

本文提出用噪声对比估计(NCE)对双向神经网络语言模型(NNLM)进行训练。实验是在PTB数据集上进行的。结果表明,NCE训练的双向NNLM优于传统的最大似然训练的训练。但仍然(遗憾的是),它没有超过基线单向NNLM。

英文摘要:

We propose to train bi-directional neural network language model(NNLM) with noise contrastive estimation(NCE). Experiments are conducted on a rescore task on the PTB data set. It is shown that NCE-trained bi-directional NNLM outperformed the one trained by conventional maximum likelihood training. But still(regretfully), it did not out-perform the baseline uni-directional NNLM.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑