KazQAD：哈萨克语开放域问答数据集
KazQAD: Kazakh Open-Domain Question Answering Dataset
AI总结：
本文构建了哈萨克语开放域问答数据集 KazQAD，包含约 6000 个问题、1.2 万条段落相关性判断和 80 多万个维基百科段落，并提供基线检索与阅读模型结果，揭示了低资源语言问答仍有较大提升空间。
摘要：
我们介绍 KazQAD——一个哈萨克语开放域问答（ODQA）数据集——它既可用于阅读理解设置和完整的开放域问答设置，也可用于信息检索实验。KazQAD 包含略少于 6,000 个带有抽取式短答案的唯一问题，以及近 12,000 个段落级相关性判断。我们结合使用机器翻译、维基百科搜索和内部人工标注，以确保标注效率和数据质量。问题来自两个来源：来自 Natural Questions（NQ）数据集的翻译条目（仅用于训练）和原始的哈萨克语统一国家测试（UNT）考试题目（用于开发和测试）。随附的文本语料库包含来自哈萨克语维基百科的 80 多万个段落。作为补充数据集，我们发布了约 61,000 个来自 NQ 数据集的、已机器翻译成哈萨克语的问题-段落-答案三元组。我们开发了基线检索器和阅读器，在检索（NDCG@10 = 0.389，MRR = 0.382）、阅读理解（EM = 38.5，F1 = 54.2）和完整开放域问答（EM = 17.8，F1 = 28.7）设置中取得了合理的分数。然而，这些结果明显低于英语问答集合的最新结果，我们认为仍应有很大的改进空间。我们还表明，当前 OpenAI 的 ChatGPTv3.5 无法在闭卷设置中以可接受的质量回答 KazQAD 测试问题。该数据集在知识共享许可（CC BY-SA）下免费提供，网址为 https://github.com/IS2AI/KazQAD。
英文摘要：
We introduce KazQAD -- a Kazakh open-domain question answering (ODQA) dataset -- that can be used in both reading comprehension and full ODQA settings, as well as for information retrieval experiments. KazQAD contains just under 6,000 unique questions with extracted short answers and nearly 12,000 passage-level relevance judgements. We use a combination of machine translation, Wikipedia search, and in-house manual annotation to ensure annotation efficiency and data quality. The questions come from two sources: translated items from the Natural Questions (NQ) dataset (only for training) and the original Kazakh Unified National Testing (UNT) exam (for development and testing). The accompanying text corpus contains more than 800,000 passages from the Kazakh Wikipedia. As a supplementary dataset, we release around 61,000 question-passage-answer triples from the NQ dataset that have been machine-translated into Kazakh. We develop baseline retrievers and readers that achieve reasonable scores in retrieval (NDCG@10 = 0.389 MRR = 0.382), reading comprehension (EM = 38.5 F1 = 54.2), and full ODQA (EM = 17.8 F1 = 28.7) settings. Nevertheless, these results are substantially lower than state-of-the-art results for English QA collections, and we think that there should still be ample room for improvement. We also show that the current OpenAI's ChatGPTv3.5 is not able to answer KazQAD test questions in the closed-book setting with acceptable quality. The dataset is freely available under the Creative Commons licence (CC BY-SA) at https://github.com/IS2AI/KazQAD.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
