KQA Pro：一个基于知识库的复杂问答数据集
KQA Pro: A Dataset with Explicit Compositional Programs for Complex Question Answering over Knowledge Base
摘要：
基于知识库的复杂问答（Complex KBQA）是一个具有挑战性的问题，因为它需要多种组合推理能力，如多跳推理、属性比较、集合运算等。现有的基准测试存在以下不足：1）只提供问答对，没有明确的推理过程; 2）问题的多样性和规模性较差。为此，我们介绍了KQA Pro，这是一个用于复杂KBQA的数据集，包括约12万个不同的自然语言问题。我们引入了一个组合和可解释的程序设计语言KoPL来表示复杂问题的推理过程。对于每个问题，我们提供了相应的KoPL程序和SPARQL查询，使KQA Pro服务于KBQA和语义解析任务。实验结果表明，SOTA KBQA方法不能在KQA Pro上取得有希望的结果，这表明KQA Pro具有挑战性，复杂KBQA需要进一步的研究工作。我们还将KQA Pro视为测试多种推理技能的诊断数据集，对现有模型进行全面评估，并讨论复杂KBQA的进一步方向。我们的代码和数据集可以从https://github.com/shijx12/KQAPro_Baselines获得。
英文摘要：
Complex question answering over knowledge base (Complex KBQA) is challenging because it requires various compositional reasoning capabilities, such as multi-hop inference, attribute comparison, set operation. Existing benchmarks have some shortcomings that limit the development of Complex KBQA: 1) they only provide QA pairs without explicit reasoning processes; 2) questions are poor in diversity or scale. To this end, we introduce KQA Pro, a dataset for Complex KBQA including ~120K diverse natural language questions. We introduce a compositional and interpretable programming language KoPL to represent the reasoning process of complex questions. For each question, we provide the corresponding KoPL program and SPARQL query, so that KQA Pro serves for both KBQA and semantic parsing tasks. Experimental results show that SOTA KBQA methods cannot achieve promising results on KQA Pro as on current datasets, which suggests that KQA Pro is challenging and Complex KBQA requires further research efforts. We also treat KQA Pro as a diagnostic dataset for testing multiple reasoning skills, conduct a thorough evaluation of existing models and discuss further directions for Complex KBQA. Our codes and datasets can be obtained from https://github.com/shijx12/KQAPro_Baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
