arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2212.10189cs.CLcs.AI
收藏

我有回答的知识吗?探究知识库问题的可回答性

Do I have the Knowledge to Answer? Investigating Answerability of Knowledge Base Questions

TCS Research(塔塔咨询服务公司研究院);Indian Institute of Technology, Delhi(德里印度理工学院)

Mayur Patidar, Prayushi Faldu, Avinash Singh, Lovekesh Vig, Indrajit Bhattacharya, Mausam

AI总结:

本文针对知识库问答(KBQA)中的不可回答性问题,创建了新基准数据集GrailQAbility,实验发现现有最先进模型在处理不可回答问题时性能下降且存在误判,凸显了提升系统鲁棒性的需求。

摘要:

在基于知识库回答自然语言问题时,事实缺失、模式不完整和范围有限自然会导致许多问题无法回答。尽管可回答性在其他QA设置中已被探讨,但尚未在基于知识库的问答(KBQA)中进行研究。我们创建了GrailQAbility,一个新的带有不可回答性的基准KBQA数据集,方法是首先识别导致问题无法回答的各种KB不完整形式,然后系统地改编GrailQA(一个仅包含可回答问题的流行KBQA数据集)。在对三个最先进的KBQA模型进行实验时,我们发现即使在对不可回答问题进行了适当调整后,所有三个模型的性能都有所下降。此外,这些模型常常因错误原因检测到不可回答性,并发现特定形式的不可回答性特别难以处理。这强调了进一步研究以使KBQA系统对不可回答性具有鲁棒性的必要性。

英文摘要:

When answering natural language questions over knowledge bases, missing facts, incomplete schema and limited scope naturally lead to many questions being unanswerable. While answerability has been explored in other QA settings, it has not been studied for QA over knowledge bases (KBQA). We create GrailQAbility, a new benchmark KBQA dataset with unanswerability, by first identifying various forms of KB incompleteness that make questions unanswerable, and then systematically adapting GrailQA (a popular KBQA dataset with only answerable questions). Experimenting with three state-of-the-art KBQA models, we find that all three models suffer a drop in performance even after suitable adaptation for unanswerable questions. In addition, these often detect unanswerability for wrong reasons and find specific forms of unanswerability particularly difficult to handle. This underscores the need for further research in making KBQA systems robust to unanswerability

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑