arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.01382cs.CL
收藏

面向长尾知识的自动问答生成

Automatic Question-Answer Generation for Long-Tail Knowledge

Carnegie Mellon University(卡内基梅隆大学)

Rohan Kumar, Youngmin Kim, Sunitha Ravi, Haitian Sun, Christos Faloutsos, Ruslan Salakhutdinov, Minji Yoon

AI总结:

针对 LLM 缺乏尾部实体 QA 评测数据的问题,本文提出自动生成长尾知识专用 QA 数据集的方法,并比较预训练 LLM 在有无 Wikipedia、Wikidata 等外部资源时的表现。

摘要:

预训练大语言模型(LLM)因用于解决开放域问答(QA)而受到广泛关注。尽管它们在回答与常识相关的问题时表现出较高准确率,但 LLM 在学习不常见的长尾知识(尾部实体)时会遇到困难。由于人工构建 QA 数据集需要大量人力资源,现有 QA 数据集的类型有限,导致缺乏用于研究 LLM 在尾部实体上表现的数据集。本文提出一种自动方法,用于生成面向尾部实体的专用 QA 数据集,并阐述相关研究挑战。我们在新生成的长尾 QA 数据集上使用预训练 LLM 开展大量实验,比较其在使用与不使用包括 Wikipedia 和 Wikidata 知识图谱在内的外部资源时的性能。

英文摘要:

Pretrained Large Language Models (LLMs) have gained significant attention for addressing open-domain Question Answering (QA). While they exhibit high accuracy in answering questions related to common knowledge, LLMs encounter difficulties in learning about uncommon long-tail knowledge (tail entities). Since manually constructing QA datasets demands substantial human resources, the types of existing QA datasets are limited, leaving us with a scarcity of datasets to study the performance of LLMs on tail entities. In this paper, we propose an automatic approach to generate specialized QA datasets for tail entities and present the associated research challenges. We conduct extensive experiments by employing pretrained LLMs on our newly generated long-tail QA datasets, comparing their performance with and without external resources including Wikipedia and Wikidata knowledge graphs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑