FIRE：基于迭代检索与验证的事实核查
FIRE: Fact-checking with Iterative Retrieval and Verification
AI总结：
提出FIRE智能体框架，通过迭代检索与验证，依据置信度决定是否弃权（不执行）或生成搜索查询，在略优性能下大幅降低LLM和搜索成本。
摘要：
对长文本进行事实核查具有挑战性，因此通常的做法是将其分解为多个原子声明。对这些原子声明进行事实核查的典型方法涉及检索固定数量的证据，然后进行验证步骤。然而，这种方法通常成本效益不高，因为它未能充分利用验证模型对声明的内部知识，也未能复现人类搜索策略中的迭代推理过程。为解决这些局限性，我们提出了FIRE，一种新颖的基于智能体的框架，以迭代方式整合证据检索与声明验证。具体而言，FIRE采用统一机制，根据其对当前判断的置信度，决定是提供最终答案还是生成后续搜索查询。我们将FIRE与其他强大的事实核查框架进行比较，发现它在取得略优性能的同时，将大型语言模型（LLM）成本平均降低7.6倍，搜索成本降低16.5倍。这些结果表明，FIRE在大规模事实核查操作中具有应用前景。我们的代码可在https://github.com/mbzuai-nlp/fire.git获取。
英文摘要：
Fact-checking long-form text is challenging, and it is therefore common practice to break it down into multiple atomic claims. The typical approach to fact-checking these atomic claims involves retrieving a fixed number of pieces of evidence, followed by a verification step. However, this method is usually not cost-effective, as it underutilizes the verification model's internal knowledge of the claim and fails to replicate the iterative reasoning process in human search strategies. To address these limitations, we propose FIRE, a novel agent-based framework that integrates evidence retrieval and claim verification in an iterative manner. Specifically, FIRE employs a unified mechanism to decide whether to provide a final answer or generate a subsequent search query, based on its confidence in the current judgment. We compare FIRE with other strong fact-checking frameworks and find that it achieves slightly better performance while reducing large language model (LLM) costs by an average of 7.6 times and search costs by 16.5 times. These results indicate that FIRE holds promise for application in large-scale fact-checking operations. Our code is available at https://github.com/mbzuai-nlp/fire.git.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
