利用对比学习改进真实场景事实核查的检索性能
Contrastive Learning to Improve Retrieval for Real-world Fact Checking
AI总结：
针对事实核查中证据检索需推理的瓶颈，提出基于多训练信号对比微调的CFR重排器，在AVeriTeC上准确率提升6%，且增益可迁移至多个相关数据集。
摘要：
近期的事实核查研究聚焦于一种真实场景：模型会整合从网络检索到的证据来判定主张的真实性。该流程的一个瓶颈在于相关证据的检索：传统方法可能会返回与主张直接相关的文档，但复杂主张的事实核查需要更多推理。例如，一份关于疫苗研发过程的文档，即便没有直接回应相关主张，也与解答“疫苗可能含有什么成分”这类主张相关。我们提出了对比事实核查重排器（Contrastive Fact-Checking Reranker, CFR）——一种适用于该场景的改进型检索器。我们利用AVeriTeC数据集（该数据集中为主张标注了子问题，且子问题配有来自证据文档的人工撰写答案），基于多种训练信号以对比目标微调Contriever模型，这些训练信号包括GPT-4的蒸馏结果、子问题答案的评估结果，以及数据集中的金标标签。我们在检索任务和主张真实性的端到端判定任务上评估了我们的模型。在AVeriTeC数据集上，我们发现真实性分类准确率提升了6%。我们还表明，该性能增益可迁移到FEVER、ClaimDecomp、HotpotQA以及一个需要检索器进行推理的合成数据集上。
英文摘要：
Recent work on fact-checking addresses a realistic setting where models incorporate evidence retrieved from the web to decide the veracity of claims. A bottleneck in this pipeline is in retrieving relevant evidence: traditional methods may surface documents directly related to a claim, but fact-checking complex claims requires more inferences. For instance, a document about how a vaccine was developed is relevant to addressing claims about what it might contain, even if it does not address them directly. We present Contrastive Fact-Checking Reranker (CFR), an improved retriever for this setting. By leveraging the AVeriTeC dataset, which annotates subquestions for claims with human written answers from evidence documents, we fine-tune Contriever with a contrastive objective based on multiple training signals, including distillation from GPT-4, evaluating subquestion answers, and gold labels in the dataset. We evaluate our model on both retrieval and end-to-end veracity judgments about claims. On the AVeriTeC dataset, we find a 6\% improvement in veracity classification accuracy. We also show our gains can be transferred to FEVER, ClaimDecomp, HotpotQA, and a synthetic dataset requiring retrievers to make inferences.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
