arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1603.08148cs.CLcs.LGcs.NE
收藏

指向未知的单词

Pointing the Unknown Words

Université de Montréal(蒙特利尔大学);IBM T.J. Watson Research(IBM T.J.沃森研究中心)

Caglar Gulcehre, Sungjin Ahn, Ramesh Nallapati, Bowen Zhou, Yoshua Bengio

摘要:

稀有和未知词的问题是一个重要的问题,可能会影响许多NLP系统的性能,包括传统的基于计数的模型和深度学习模型。我们提出了一种新的方法来处理罕见的和看不见的话的神经网络模型使用注意。我们的模型使用两个softmax层来预测条件语言模型中的下一个单词:一个预测单词在源句子中的位置,另一个预测候选词汇表中的单词。在每个时间步,使用哪个softmax层的决定由以上下文为条件的MLP自适应地做出。我们的工作动机来自一个心理学证据,即人类自然倾向于在不知道物体名称的情况下指向上下文或环境中的物体。我们观察到两个任务的改进,神经机器翻译的Europarl英语到法语的平行语料库和文本摘要的Gigaword数据集使用我们提出的模型。

英文摘要:

The problem of rare and unknown words is an important issue that can potentially influence the performance of many NLP systems, including both the traditional count-based and the deep learning models. We propose a novel way to deal with the rare and unseen words for the neural network models using attention. Our model uses two softmax layers in order to predict the next word in conditional language models: one predicts the location of a word in the source sentence, and the other predicts a word in the shortlist vocabulary. At each time-step, the decision of which softmax layer to use choose adaptively made by an MLP which is conditioned on the context.~We motivate our work from a psychological evidence that humans naturally have a tendency to point towards objects in the context or the environment when the name of an object is not known.~We observe improvements on two tasks, neural machine translation on the Europarl English to French parallel corpora and text summarization on the Gigaword dataset using our proposed model.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑