arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.08735cs.CL
收藏

阅读理解的实例包学习

Learning with Instance Bundles for Reading Comprehension

University of California, Irvine(加利福尼亚大学尔湾分校);Allen Institute for Artificial Intelligence(艾伦人工智能研究所)

Dheeru Dua, Pradeep Dasigi, Sameer Singh, Matt Gardner

摘要:

在训练大多数现代阅读理解模型时,与上下文相关的所有问题都被视为彼此独立的。然而,密切相关的问题及其相应的答案并不是独立的,利用这些关系可以为模型提供强有力的监督信号。借鉴对比估计的思想,我们介绍了几种新的监督技术,比较多个相关实例的问答分数。具体来说,我们将这些分数在不同邻域的密切对比的问题和/或答案中进行归一化,添加除了传统的最大似然估计之外使用的另一个交叉熵损失项。我们的技术需要捆绑相关的问题-答案对,我们可以从现有数据中挖掘,也可以使用各种自动化算法创建。我们在两个数据集(HotpotQA和ROPES)上实证了使用实例包进行训练的有效性,显示出高达11%的准确率绝对增益。

英文摘要:

When training most modern reading comprehension models, all the questions associated with a context are treated as being independent from each other. However, closely related questions and their corresponding answers are not independent, and leveraging these relationships could provide a strong supervision signal to a model. Drawing on ideas from contrastive estimation, we introduce several new supervision techniques that compare question-answer scores across multiple related instances. Specifically, we normalize these scores across various neighborhoods of closely contrasting questions and/or answers, adding another cross entropy loss term that is used in addition to traditional maximum likelihood estimation. Our techniques require bundles of related question-answer pairs, which we can either mine from within existing data or create using various automated heuristics. We empirically demonstrate the effectiveness of training with instance bundles on two datasets -- HotpotQA and ROPES -- showing up to 11% absolute gains in accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑