arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2004.03238cs.CLcs.AIcs.LG
收藏

用变分的应答对生成方法提高QA模型对挑战集的鲁棒性

Improving the Robustness of QA Models to Challenge Sets with Variational Question-Answer Pair Generation

The University of Tokyo(东京大学);National Institute of Informatics(国立信息学研究所)

Kazutoshi Shinoda, Saku Sugawara, Akiko Aizawa

摘要:

阅读理解的问答(QA)模型在分布测试集上已经达到了人类水平的准确性。然而,它们已被证明缺乏对挑战集的鲁棒性,挑战集的分布与训练集的分布不同。现有的数据增强方法通过简单地用从与挑战集相同的分布中采样的合成示例来增强训练集来缓解这个问题。然而,这些方法假设挑战集的分布是先验已知的,使得它们不太适用于看不见的挑战集。在这项研究中,我们专注于问答对生成(QAG),以减轻这个问题。虽然大多数现有的QAG方法的目的是提高合成的例子的质量,我们推测,多样性促进QAG可以减轻训练集的稀疏性,并导致更好的鲁棒性。我们提出了一个变分QAG模型,从一个段落中产生多个不同的QA对。实验结果表明,该方法可以提高12个挑战集的准确率,以及分布准确率。我们的代码和数据可在https://github.com/KazutoshiShinoda/VQAG上获得。

英文摘要:

Question answering (QA) models for reading comprehension have achieved human-level accuracy on in-distribution test sets. However, they have been demonstrated to lack robustness to challenge sets, whose distribution is different from that of training sets. Existing data augmentation methods mitigate this problem by simply augmenting training sets with synthetic examples sampled from the same distribution as the challenge sets. However, these methods assume that the distribution of a challenge set is known a priori, making them less applicable to unseen challenge sets. In this study, we focus on question-answer pair generation (QAG) to mitigate this problem. While most existing QAG methods aim to improve the quality of synthetic examples, we conjecture that diversity-promoting QAG can mitigate the sparsity of training sets and lead to better robustness. We present a variational QAG model that generates multiple diverse QA pairs from a paragraph. Our experiments show that our method can improve the accuracy of 12 challenge sets, as well as the in-distribution accuracy. Our code and data are available at https://github.com/KazutoshiShinoda/VQAG.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑