arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.01461cs.CL
收藏

检索增强开放域问答中的可回答性

Answerability in Retrieval-Augmented Open-Domain Question Answering

University of Tartu(塔尔图大学);University of Edinburgh(爱丁堡大学);University of Copenhagen(哥本哈根大学)

Rustam Abdumalikov, Pasquale Minervini, Yova Kementchedjhieva

AI总结:

针对ODQA检索中高语义重叠的不相关文本识别难题,本文利用SQuAD 2.0不可回答配对训练模型,将准确率从随机配对策略的1%提升至约100%。

摘要:

开放域问答(ODQA)检索系统的性能可能表现出次优行为,提供具有不同程度不相关性的文本摘录。遗憾的是,许多现有ODQA数据集缺乏专门针对识别不相关文本摘录的样本。以往弥补这一空白的尝试依赖于一种简单方法,即将问题与随机文本摘录配对。本文旨在研究使用这种随机策略训练的模型的有效性,揭示其在泛化到具有高语义重叠的不相关文本摘录方面的一个重要局限。因此,我们观察到预测准确率大幅下降,从98%降至1%。为解决这一局限,我们发现了一种训练模型识别此类摘录的高效方法。通过利用SQuAD 2.0数据集中的不可回答配对,我们的模型在面对这些具有挑战性的文本摘录时取得了近乎完美(约100%)的准确率。

英文摘要:

The performance of Open-Domain Question Answering (ODQA) retrieval systems can exhibit sub-optimal behavior, providing text excerpts with varying degrees of irrelevance. Unfortunately, many existing ODQA datasets lack examples specifically targeting the identification of irrelevant text excerpts. Previous attempts to address this gap have relied on a simplistic approach of pairing questions with random text excerpts. This paper aims to investigate the effectiveness of models trained using this randomized strategy, uncovering an important limitation in their ability to generalize to irrelevant text excerpts with high semantic overlap. As a result, we observed a substantial decrease in predictive accuracy, from 98% to 1%. To address this limitation, we discovered an efficient approach for training models to recognize such excerpts. By leveraging unanswerable pairs from the SQuAD 2.0 dataset, our models achieve a nearly perfect (~100%) accuracy when confronted with these challenging text excerpts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑