直面事实！评估用于专业事实核查的基于RAG的流水线
Face the Facts! Evaluating RAG-based Pipelines for Professional Fact-Checking
AI总结：
该研究遵循专业事实核查规范，解除现有基于RAG的自动化事实核查流水线的部分限制，对RAG类方法生成主张真实性判定的效果开展基准测试，揭示了不同检索技术、模型规模及训练方式在多维度的性能差异。
摘要：
自然语言处理与生成系统近期展现出补充并简化专业事实核查人员这一成本高昂且耗时工作的潜力。在本研究中，我们解除了当前基于检索增强生成（Retrieval-Augmented Generation, RAG）范式的最先进自动化事实核查流水线的若干限制。我们的目标是遵循专业事实核查规范，对用于生成判定结果（即论述主张真实性的短文）的基于RAG的方法进行基准测试，在风格复杂的主张以及异构但可靠的知识库上对其展开评估。我们的研究结果呈现出复杂的局面：例如，基于大语言模型（LLM）的检索器性能优于其他检索技术，但它们在处理异构知识库时仍存在困难；更大的模型在判定结果忠实度上表现出色，而较小的模型则具备更好的上下文遵循性；人工评估显示，零样本和少样本方法在信息丰富度上更受青睐，微调模型则在情感一致性上表现更优。
英文摘要：
Natural Language Processing and Generation systems have recently shown the potential to complement and streamline the costly and time-consuming job of professional fact-checkers. In this work, we lift several constraints of current state-of-the-art pipelines for automated fact-checking based on the Retrieval-Augmented Generation (RAG) paradigm. Our goal is to benchmark, following professional fact-checking practices, RAG-based methods for the generation of verdicts - i.e., short texts discussing the veracity of a claim - evaluating them on stylistically complex claims and heterogeneous, yet reliable, knowledge bases. Our findings show a complex landscape, where, for example, LLM-based retrievers outperform other retrieval techniques, though they still struggle with heterogeneous knowledge bases; larger models excel in verdict faithfulness, while smaller models provide better context adherence, with human evaluations favouring zero-shot and one-shot approaches for informativeness, and fine-tuned models for emotional alignment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
