arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.25501cs.CL
收藏

对检测虚假新闻最流行方法的实验比较

An Experimental Comparison of the Most Popular Approaches to Fake News Detection

Dipartimento di Ingegneria dell'Informazione, University of Pisa(比萨大学信息工程系);Dipartimento di Informatica, University of Pisa(比萨大学计算机科学系)

Pietro Dell'Oglio, Alessandro Bondielli, Francesco Marcelloni, Lucia C. Passaro

AI总结:

本文对比了12种主流虚假新闻检测方法,评估了不同数据集下的分类性能,发现领域内模型表现良好但泛化能力差,跨领域架构和LLM在零样本学习中表现突出。

摘要:

近年来,虚假新闻检测在公共讨论和科学研究中受到越来越多关注。尽管检测技术有所进步,但虚假信息的产生和传播变得更加复杂,受大型语言模型(LLMs)和社交媒体放大效应驱动。本文对12种代表性的虚假新闻检测方法进行了批判性评估,涵盖传统机器学习、深度学习、transformers以及专门的跨领域架构。这些方法在10个公开数据集上进行评估,数据集在类型、来源、主题和标注逻辑上各不相同。通过将标签统一为“真实”和“虚假”,将文本-only英文虚假新闻检测作为二分类任务进行评估。我们承认不同数据集的标签语义不同,标签统一不可避免地会丢失一些语义细微差别。每个数据集被视为一个独立领域。我们进行了领域内、多领域和跨领域实验,以模拟涉及领域转移和分布外数据的现实场景。微调模型在领域内表现良好,但泛化能力较差。跨领域架构可以缩小这一差距,但需要大量数据,而LLM通过零样本和少样本学习提供了有前景的替代方案。鉴于固有数据集偏差和可能的预训练暴露,结果应被视为在此英文文本-only协议下的鲁棒性评估。

英文摘要:

In recent years, fake news detection has received increasing attention in public debate and scientific research. Despite advances in detection techniques, the production and spread of false information have become more sophisticated, driven by Large Language Models (LLMs) and the amplification power of social media. We present a critical assessment of 12 representative fake news detection approaches, spanning traditional machine learning, deep learning, transformers, and specialized cross-domain architectures. We evaluate these methods on 10 publicly available datasets differing in genre, source, topic, and labeling rationale. We address text-only English fake news detection as a binary classification task by harmonizing labels into "Real" and "Fake" to ensure a consistent evaluation protocol. We acknowledge that label semantics vary across datasets and that harmonization inevitably removes such semantic nuances. Each dataset is treated as a distinct domain. We conduct in-domain, multi-domain and cross-domain experiments to simulate real-world scenarios involving domain shift and out-of-distribution data. Fine-tuned models perform well in-domain but struggle to generalize. Cross-domain architectures can reduce this gap but are data-hungry, while LLMs offer a promising alternative through zero- and few-shot learning. Given inherent dataset confounds and possible pre-training exposure, results should be interpreted as robustness evaluations within this English, text-only protocol.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑