arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1606.02858cs.CLcs.AI
收藏

对CNN/Daily Mail阅读理解任务的彻底检查

A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task

Stanford University(斯坦福大学)

Danqi Chen, Jason Bolton, Christopher D. Manning

摘要:

使计算机能够理解文档,以便它能够回答理解问题是NLP的核心目标,但尚未解决。阻碍机器学习系统解决这个问题的一个关键因素是人类注释数据的可用性有限。Hermann et al.(2015)试图通过将CNN和Daily Mail的新闻文章与其总结的要点配对来创建超过一百万个训练样本来解决这个问题,并表明神经网络可以被训练以在这个任务上提供良好的性能。在本文中,我们对这一新的阅读理解任务进行了深入的研究。我们的主要目的是了解做好这项任务需要什么深度的语言理解。我们一方面通过对一小部分问题进行仔细的手工分析来实现这一点,另一方面通过展示简单,精心设计的系统可以在这两个数据集上获得73.6%和76.6%的准确率,超过当前最先进的结果7-10%,接近我们认为的性能上限。

英文摘要:

Enabling a computer to understand a document so that it can answer comprehension questions is a central, yet unsolved goal of NLP. A key factor impeding its solution by machine learned systems is the limited availability of human-annotated data. Hermann et al. (2015) seek to solve this problem by creating over a million training examples by pairing CNN and Daily Mail news articles with their summarized bullet points, and show that a neural network can then be trained to give good performance on this task. In this paper, we conduct a thorough examination of this new reading comprehension task. Our primary aim is to understand what depth of language understanding is required to do well on this task. We approach this from one side by doing a careful hand-analysis of a small subset of the problems and from the other by showing that simple, carefully designed systems can obtain accuracies of 73.6% and 76.6% on these two datasets, exceeding current state-of-the-art results by 7-10% and approaching what we believe is the ceiling for performance on this task.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑