arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1608.00318cs.CLcs.LG
收藏

一种神经知识语言模型

A Neural Knowledge Language Model

Université de Montréal(蒙特利尔大学);Handong Global University(韩东全球大学)

Sungjin Ahn, Heeyoul Choi, Tanel Pärnamaa, Yoshua Bengio

摘要:

当前的语言模型在编码和解码事实知识的能力方面具有显著的局限性。这主要是因为他们从统计共现中获得这些知识,尽管大多数知识词很少被观察到。在本文中,我们提出了一个神经知识语言模型(NKLM),它结合了由知识图谱提供的符号知识与RNN语言模型。通过预测要生成的词是否具有潜在事实,模型可以通过从预测事实的描述复制来生成这样的知识相关词。在实验中,我们表明,NKLM显着提高了性能,同时产生的未知词的数量少得多。

英文摘要:

Current language models have a significant limitation in the ability to encode and decode factual knowledge. This is mainly because they acquire such knowledge from statistical co-occurrences although most of the knowledge words are rarely observed. In this paper, we propose a Neural Knowledge Language Model (NKLM) which combines symbolic knowledge provided by the knowledge graph with the RNN language model. By predicting whether the word to generate has an underlying fact or not, the model can generate such knowledge-related words by copying from the description of the predicted fact. In experiments, we show that the NKLM significantly improves the performance while generating a much smaller number of unknown words.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑