arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.09754cs.AI
收藏

EX-FEVER:一个用于多跳可解释事实验证的数据集

EX-FEVER: A Dataset for Multi-hop Explainable Fact Verification

School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);ByteDance Inc.(字节跳动公司)

Huanhuan Ma, Weizhi Xu, Yifan Wei, Liuji Chen, Liang Wang, Qiang Liu, Shu Wu, Liang Wang

AI总结:

本文构建包含6万余条2跳与3跳推理声明的 EX-FEVER 数据集,为多跳事实验证提供真实性标签与推理路径解释,并通过基线系统验证其价值。

摘要:

事实验证旨在基于若干证据自动探查声明的真实性。现有工作总是致力于提升准确率,更不必说可解释性这一事实验证系统的关键能力。在复杂的多跳场景中构建可解释的事实验证系统,始终因缺乏相关的高质量数据集而受阻。以往的数据集要么存在过度简化的问题,要么未能纳入对可解释性的必要考量。为解决这一问题,我们提出了 EX-FEVER,这是一个开创性的多跳可解释事实验证数据集。该数据集包含超过 60,000 条涉及 2 跳和 3 跳推理的声明,每条声明均通过总结和修改来自超链接 Wikipedia 文档的信息而创建。每个实例都配有一个真实性标签和一段解释,该解释概述了支持真实性分类的推理路径。此外,我们在 EX-FEVER 数据集上展示了一个新颖的基线系统,呈现了文档检索、解释生成和声明验证,并验证了该数据集的重要性。进一步地,我们强调了在事实验证任务中利用大型语言模型的潜力。我们希望本数据集能够提供充分机会,以探索自然语言解释在事实验证领域中的融合,从而作出重要贡献。

英文摘要:

Fact verification aims to automatically probe the veracity of a claim based on several pieces of evidence. Existing works are always engaging in accuracy improvement, let alone explainability, a critical capability of fact verification systems. Constructing an explainable fact verification system in a complex multi-hop scenario is consistently impeded by the absence of a relevant, high-quality dataset. Previous datasets either suffer from excessive simplification or fail to incorporate essential considerations for explainability. To address this, we present EXFEVER, a pioneering dataset for multi-hop explainable fact verification. With over 60,000 claims involving 2-hop and 3-hop reasoning, each is created by summarizing and modifying information from hyperlinked Wikipedia documents. Each instance is accompanied by a veracity label and an explanation that outlines the reasoning path supporting the veracity classification. Additionally, we demonstrate a novel baseline system on our EX-FEVER dataset, showcasing document retrieval, explanation generation, and claim verification, and validate the significance of our dataset. Furthermore, we highlight the potential of utilizing Large Language Models in the fact verification task. We hope our dataset could make a significant contribution by providing ample opportunities to explore the integration of natural language explanations in the domain of fact verification.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑