通过概率确定性与一致性进行大型语言模型的事实核查
Fact-Checking with Large Language Models via Probabilistic Certainty and Consistency
AI总结：
PCC通过概率确定性与一致性框架，自适应决定LLM是否依赖内部知识或启动检索，以提升事实核查的准确性和效率。
摘要：
大型语言模型（LLMs）越来越多地用于需要事实准确性应用，但其输出常包含幻觉响应。尽管事实核查可以缓解这些错误，但现有方法通常 indiscriminately检索外部证据，忽视模型内部知识，可能引入无关噪声。此外，现有系统缺乏解决模型推理特定不确定性的针对性机制。受人类事实核查方式的启发，我们主张LLMs应根据对给定主张的信心，自适应决定是否依赖内部知识或启动检索。我们引入概率确定性与一致性（PCC），一种通过联合建模LLM的概率确定性与推理一致性来估计事实信心的框架。这些信心信号使模型能够采用自适应验证策略：当自信时直接作答，当不确定或不一致时触发针对性检索，当模糊性高时升级至深度搜索。我们的信心引导路由机制确保检索仅在必要时被触发，提高了效率和可靠性。在三个具有挑战性的基准测试中，广泛实验表明PCC在不确定性量化方面优于口头化信心，并且在强LLM事实核查基线中表现一贯优异。此外，我们证明PCC在各种LLM之间具有良好的泛化能力。
英文摘要：
Large language models (LLMs) are increasingly used in applications requiring factual accuracy, yet their outputs often contain hallucinated responses. While fact-checking can mitigate these errors, existing methods typically retrieve external evidence indiscriminately, overlooking the model's internal knowledge and potentially introducing irrelevant noise. Moreover, current systems lack targeted mechanisms to resolve specific uncertainties in the model's reasoning. Inspired by how humans fact-check, we argue that LLMs should adaptively decide whether to rely on internal knowledge or initiate retrieval based on their confidence in a given claim. We introduce Probabilistic Certainty and Consistency (PCC), a framework that estimates factual confidence by jointly modeling an LLM's probabilistic certainty and reasoning consistency. These confidence signals enable an adaptive verification strategy: the model answers directly when confident, triggers targeted retrieval when uncertain or inconsistent, and escalates to deep search when ambiguity is high. Our confidence-guided routing mechanism ensures that retrieval is invoked only when necessary, improving both efficiency and reliability. Extensive experiments across three challenging benchmarks show that PCC achieves better uncertainty quantification than verbalized confidence and consistently outperforms strong LLM-based fact-checking baselines. Furthermore, we demonstrate that PCC generalizes well across various LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
