CHECKWHY：基于论证结构的因果事实核查
CHECKWHY: Causal Fact Verification via Argument Structure
AI总结：
本文提出 CheckWhy 数据集，针对因果事实核查任务，通过声明-证据-论证结构三元组检验因果关系的真实性，实验表明引入论证结构对核查至关重要，但现有模型生成论证结构仍具挑战。
摘要：
随着事实核查任务复杂度的增加，对“深思熟虑”推理能力的关注日益提升。然而，近期的事实核查基准主要集中于核查声明中狭小范围的语义事实，且缺乏明确的逻辑推理过程。本文提出 CheckWhy，一个针对新型因果事实核查任务的挑战性数据集：通过严谨的推理步骤核查声明中因果关系的真实性。CheckWhy 包含超过 19K 个“why”声明-证据-论证结构三元组，标注为 supports、refutes 和 not enough info。每个论证结构由相连证据组成，表示从基础证据出发逐步推进至声明成立的推理过程。通过对 state-of-the-art 模型的广泛实验，验证了将论证结构纳入因果事实核查的重要性。此外，对论证结构生成的自动与人工评估显示，fine-tuned 模型或 Chain-of-Thought prompted LLMs 难以生成令人满意的论证结构，为未来改进留下较大空间。
英文摘要：
With the growing complexity of fact verification tasks, the concern with "thoughtful" reasoning capabilities is increasing. However, recent fact verification benchmarks mainly focus on checking a narrow scope of semantic factoids within claims and lack an explicit logical reasoning process. In this paper, we introduce CheckWhy, a challenging dataset tailored to a novel causal fact verification task: checking the truthfulness of the causal relation within claims through rigorous reasoning steps. CheckWhy consists of over 19K "why" claim-evidence-argument structure triplets with supports, refutes, and not enough info labels. Each argument structure is composed of connected evidence, representing the reasoning process that begins with foundational evidence and progresses toward claim establishment. Through extensive experiments on state-of-the-art models, we validate the importance of incorporating the argument structure for causal fact verification. Moreover, the automated and human evaluation of argument structure generation reveals the difficulty in producing satisfying argument structure by fine-tuned models or Chain-of-Thought prompted LLMs, leaving considerable room for future improvements.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
