大语言模型需要经过筛选的上下文才能可靠地进行政治事实核查——即使有推理和网络搜索
Large Language Models Require Curated Context for Reliable Political Fact-Checking -- Even with Reasoning and Web Search
AI总结：
大语言模型需通过筛选的上下文提升政治事实核查的可靠性，定制RAG系统显著提升宏F1指标。
摘要：
大语言模型（LLMs）引发了自动化端到端事实核查的希望，但先前的研究报告结果参差不齐。随着主流聊天机器人越来越多地配备推理能力和网络搜索工具——而且数百万用户已经依靠它们进行验证——严格评估迫在眉睫。我们评估了来自OpenAI、Google、Meta和DeepSeek的15种最新LLM，在超过6000个由PolitiFact核查的事实声明上，比较标准模型与推理和网络搜索变体。标准模型表现不佳，推理提供微小益处，网络搜索仅提供适度的提升，尽管事实核查信息已在网上可用。相比之下，使用PolitiFact摘要的定制RAG系统在各种模型变体上平均将宏F1提升了233%。这些发现表明，为模型提供经过筛选的高质量上下文是自动化事实核查的有希望的途径。
英文摘要：
Large language models (LLMs) have raised hopes for automated end-to-end fact-checking, but prior studies report mixed results. As mainstream chatbots increasingly ship with reasoning capabilities and web search tools -- and millions of users already rely on them for verification -- rigorous evaluation is urgent. We evaluate 15 recent LLMs from OpenAI, Google, Meta, and DeepSeek on more than 6,000 claims fact-checked by PolitiFact, comparing standard models with reasoning- and web-search variants. Standard models perform poorly, reasoning offers minimal benefits, and web search provides only moderate gains, despite fact-checks being available on the web. In contrast, a curated RAG system using PolitiFact summaries improved macro F1 by 233% on average across model variants. These findings suggest that giving models access to curated high-quality context is a promising path for automated fact-checking.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
