arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1909.03759cs.CLcs.AI
收藏

神经会话QA:学习推理与利用模式

Neural Conversational QA: Learning to Reason v.s. Exploiting Patterns

IBM Research AI(IBM研究院人工智能部门);Indian Institute of Technology (BHU)(印度理工学院(贝拿勒斯印度教大学))

Nikhil Verma, Abhishek Sharma, Dhiraj Madan, Danish Contractor, Harshit Kumar, Sachindra Joshi

摘要:

像ShARC这样的神经会话QA任务需要系统根据给定段落的内容回答问题。在研究ShARCQA任务的最新模型时,我们发现这些模型在数据集中学习了虚假的线索/模式。此外,我们表明,设计用于利用这些模式的基于神经网络的程序可以具有与神经模型相当的性能。在本文中,我们分享了我们在ShARC语料库中发现的四种模式的发现,并描述了神经模型如何利用它们。受上述发现的启发,我们创建并共享了一个修改后的数据集,该数据集具有更少的虚假模式,从而使模型能够更好地学习。

英文摘要:

Neural Conversational QA tasks like ShARC require systems to answer questions based on the contents of a given passage. On studying recent state-of-the-art models on the ShARCQA task, we found indications that the models learn spurious clues/patterns in the dataset. Furthermore, we show that a heuristic-based program designed to exploit these patterns can have performance comparable to that of the neural models. In this paper we share our findings about four types of patterns found in the ShARC corpus and describe how neural models exploit them. Motivated by the aforementioned findings, we create and share a modified dataset that has fewer spurious patterns, consequently allowing models to learn better.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑