arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1707.09098cs.AIcs.CL
收藏

MEMEN:面向机器理解的多层记忆网络嵌入

MEMEN: Multi-layer Embedding with Memory Networks for Machine Comprehension

College of Computer Science, Zhejiang University(浙江大学计算机科学与技术学院);Eigen Technologies

Boyuan Pan, Hao Li, Zhou Zhao, Bin Cao, Deng Cai, Xiaofei He

摘要:

机器理解式问答是自然语言处理中的一个典型问题。以往的方法很少花时间在编码层的改进上,尤其是句法信息和词的名称实体的嵌入,而这对编码的质量至关重要。此外,现有的注意力方法将每个查询词表示为一个向量,或者用单个向量来表示整个查询语句,都不能正确处理查询语句中关键词的权重。在本文中,我们介绍了一种新的神经网络架构称为多层嵌入与记忆网络(MEMEMEN)的机器阅读任务。在编码层,我们采用经典的skip-gram模型对词汇的句法和语义信息进行训练,形成一种新的嵌入层。我们还提出了一个记忆网络的全方位匹配的查询和通道,以捕捉更多的关键信息。实验结果表明,该模型在Stanford问答数据集(SQuAD)上的测试结果在精度和效率方面都具有竞争力,在TriviaQA数据集上的测试结果也达到了最先进水平。

英文摘要:

Machine comprehension(MC) style question answering is a representative problem in natural language processing. Previous methods rarely spend time on the improvement of encoding layer, especially the embedding of syntactic information and name entity of the words, which are very crucial to the quality of encoding. Moreover, existing attention methods represent each query word as a vector or use a single vector to represent the whole query sentence, neither of them can handle the proper weight of the key words in query sentence. In this paper, we introduce a novel neural network architecture called Multi-layer Embedding with Memory Network(MEMEN) for machine reading task. In the encoding layer, we employ classic skip-gram model to the syntactic and semantic information of the words to train a new kind of embedding layer. We also propose a memory network of full-orientation matching of the query and passage to catch more pivotal information. Experiments show that our model has competitive results both from the perspectives of precision and efficiency in Stanford Question Answering Dataset(SQuAD) among all published results and achieves the state-of-the-art results on TriviaQA dataset.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑