利用野外检索到的证据进行复杂声明验证
Complex Claim Verification with Evidence Retrieved in the Wild
AI总结：
提出首个从网络检索声明提出前的原始证据并自动核查真实复杂声明的五阶段流水线，实验与人工评估表明其证据摘要可靠、相关并能改善真实性判断。
摘要：
证据检索是自动事实核查的核心部分。先前工作在检索中做出了脱离真实用例的简化假设：要么无法获取证据，要么可获取由人类事实核查员精选的证据，要么可获取在声明提出很久之后才出现的证据。在本工作中，我们提出首个通过从网络检索原始证据来核查真实世界声明的全自动流水线。我们将检索器限制为只搜索在声明提出之前可获得的文档，以模拟需要核查新兴声明的现实场景。我们的流水线包含五个组件：声明分解、原始文档检索、细粒度证据检索、以声明为中心的摘要生成，以及真实性判断。我们在 ClaimDecomp 数据集中的复杂政治声明上开展实验，结果表明由我们的流水线生成的聚合证据改善了真实性判断。人工评估发现，我们的系统生成的证据摘要可靠（不会产生信息幻觉），并且与回答有关声明的关键问题相关，这表明即使无法呈现完整证据集，它也能为事实核查员提供帮助。
英文摘要：
Evidence retrieval is a core part of automatic fact-checking. Prior work makes simplifying assumptions in retrieval that depart from real-world use cases: either no access to evidence, access to evidence curated by a human fact-checker, or access to evidence available long after the claim has been made. In this work, we present the first fully automated pipeline to check real-world claims by retrieving raw evidence from the web. We restrict our retriever to only search documents available prior to the claim's making, modeling the realistic scenario where an emerging claim needs to be checked. Our pipeline includes five components: claim decomposition, raw document retrieval, fine-grained evidence retrieval, claim-focused summarization, and veracity judgment. We conduct experiments on complex political claims in the ClaimDecomp dataset and show that the aggregated evidence produced by our pipeline improves veracity judgments. Human evaluation finds the evidence summary produced by our system is reliable (it does not hallucinate information) and relevant to answering key questions about a claim, suggesting that it can assist fact-checkers even when it cannot surface a complete evidence set.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
