事实比观点更难核查——基于LLM的事实核查可靠性的多语言比较分析
Facts are Harder Than Opinions -- A Multilingual, Comparative Analysis of LLM-Based Fact-Checking Reliability
AI总结：
本文构建含61,514项多语言多主题声明的可扩展数据集，评估五个主流LLM，发现语言与主题性能差距及事实型声明更易误判等可靠性问题。
摘要：
错误信息的扩散使得可扩展的自动化事实核查方案成为必需。然而，现有基准往往忽视多语言和主题多样性。本文介绍了一个新颖、可动态扩展的数据集，包含多种语言和主题的61,514项声明，并将既有数据集扩展至2024年。通过对包括GPT-4o、GPT-3.5 Turbo、LLaMA 3.1和Mixtral 8x7B在内的五个主流大型语言模型（LLM）进行综合评估，我们发现不同语言和主题之间存在显著性能差距。尽管GPT-4o总体取得最高准确率，但它拒绝对43%的声明进行分类。在所有模型中，听起来像事实的声明比观点更容易被错误分类，这揭示了一个关键脆弱性。这些发现强调了谨慎的必要性，并凸显了大规模部署基于LLM的事实核查系统所面临的挑战。
英文摘要：
The proliferation of misinformation necessitates scalable, automated fact-checking solutions. Yet, current benchmarks often overlook multilingual and topical diversity. This paper introduces a novel, dynamically extensible data set that includes 61,514 claims in multiple languages and topics, extending existing datasets up to 2024. Through a comprehensive evaluation of five prominent Large Language Models (LLMs), including GPT-4o, GPT-3.5 Turbo, LLaMA 3.1, and Mixtral 8x7B, we identify significant performance gaps between different languages and topics. While overall GPT-4o achieves the highest accuracy, it declines to classify 43% of claims. Across all models, factual-sounding claims are misclassified more often than opinions, revealing a key vulnerability. These findings underscore the need for caution and highlight challenges in deploying LLM-based fact-checking systems at scale.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
