利用LLM参数知识进行事实核查而不进行检索
Leveraging LLM Parametric Knowledge for Fact Checking without Retrieval
AI总结：
本文提出不依赖检索的事实核查任务，通过利用LLM内部表示提升事实验证性能，展示INTRA方法在多语言和长文本生成场景中的优越表现。
摘要：
可信性是基于大型语言模型（LLMs）构建的代理AI系统的核心研究挑战。为了增强可信度，来自各种来源的自然语言声明，包括人工撰写的文本、网络内容和模型输出，通常通过检索外部知识并使用LLM来验证声明与检索到的证据的忠实性来进行事实核查。因此，此类方法受到检索错误和外部数据可用性的影响，同时使模型内在的事实验证能力未被充分利用。我们提出了不进行检索的事实核查任务，专注于验证任意自然语言声明，而不考虑其来源。为了研究这种设置，我们引入了一个全面的评估框架，专注于泛化能力，测试对（i）长尾知识、（ii）声明来源的变化、（iii）多语言性和（iv）长形式生成的鲁棒性。在9个数据集、18种方法和3个模型上，我们的实验表明，基于logit的方法往往在利用内部模型表示的方法之下。基于这一发现，我们引入了INTRA，一种利用内部表示之间相互作用的方法，通过强大的泛化能力实现了最先进的性能。更广泛地说，我们的工作确立了不进行检索的事实核查作为一种有前途的研究方向，可以补充基于检索的框架，提高可扩展性，并使此类系统在训练过程中作为奖励信号，或作为生成过程中的组件使用。
英文摘要：
Trustworthiness is a core research challenge for agentic AI systems built on Large Language Models (LLMs). To enhance trust, natural language claims from diverse sources, including human-written text, web content, and model outputs, are commonly checked for factuality by retrieving external knowledge and using an LLM to verify the faithfulness of claims to the retrieved evidence. As a result, such methods are constrained by retrieval errors and external data availability, while leaving the models intrinsic fact-verification capabilities largely unused. We propose the task of fact-checking without retrieval, focusing on the verification of arbitrary natural language claims, independent of their source. To study this setting, we introduce a comprehensive evaluation framework focused on generalization, testing robustness to (i) long-tail knowledge, (ii) variation in claim sources, (iii) multilinguality, and (iv) long-form generation. Across 9 datasets, 18 methods and 3 models, our experiments indicate that logit-based approaches often underperform compared to those that leverage internal model representations. Building on this finding, we introduce INTRA, a method that exploits interactions between internal representations and achieves state-of-the-art performance with strong generalization. More broadly, our work establishes fact-checking without retrieval as a promising research direction that can complement retrieval-based frameworks, improve scalability, and enable the use of such systems as reward signals during training or as components integrated into the generation process.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
