arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.09747cs.CLcs.AI
收藏

为真相而再搜索:多轮检索增强的大型语言模型是强大的假新闻检测器

Re-Search for The Truth: Multi-round Retrieval-augmented Large Language Models are Strong Fake News Detectors

College of Computer Science and Software Engineering, Shenzhen University(深圳大学计算机与软件学院);School of Computer Science and Technology, University of Science and Technology of China(中国科学技术大学计算机科学与技术学院);Illinois Institute of Technology(伊利诺伊理工学院)

Guanghua Li, Wensheng Lu, Wei Zhang, Defu Lian, Kezhong Lu, Rui Mao, Kai Shu, Hao Liao

AI总结:

针对假新闻检测中证据不足与检索质量低的问题,提出多轮检索增强LLM框架,自动从网络提取关键证据,在三个数据集上验证了其优越性,并提供可解释裁决。

摘要:

假新闻的泛滥对政治、经济和社会整体产生了深远的影响。尽管假新闻检测方法已被用于缓解这一问题,但它们主要依赖于两个关键要素:证据的质量和相关性,以及裁决预测机制的有效性。传统方法通常从静态知识库(如维基百科)获取信息,受限于过时或不完整的数据,尤其对于新兴或罕见的主张。大型语言模型(LLMs)以其卓越的推理和生成能力,为假新闻检测开辟了新的前沿。然而,与传统方法一样,基于LLM的解决方案也面临过时和长尾知识的限制。此外,检索增强的LLM经常遇到低质量证据检索和上下文长度限制等问题。为了解决这些挑战,我们引入了一种新颖的检索增强LLM框架——这是首个能够自动且策略性地从网络来源提取关键证据以进行主张验证的框架。通过采用多轮检索策略,我们的框架确保获取充分、相关的证据,从而提升性能。在三个真实世界数据集上的全面实验验证了该框架相对于现有方法的优越性。重要的是,我们的模型不仅提供准确的裁决,还提供人类可读的解释以提高结果的可解释性。

英文摘要:

The proliferation of fake news has had far-reaching implications on politics, the economy, and society at large. While Fake news detection methods have been employed to mitigate this issue, they primarily depend on two essential elements: the quality and relevance of the evidence, and the effectiveness of the verdict prediction mechanism. Traditional methods, which often source information from static repositories like Wikipedia, are limited by outdated or incomplete data, particularly for emerging or rare claims. Large Language Models (LLMs), known for their remarkable reasoning and generative capabilities, introduce a new frontier for fake news detection. However, like traditional methods, LLM-based solutions also grapple with the limitations of stale and long-tail knowledge. Additionally, retrieval-enhanced LLMs frequently struggle with issues such as low-quality evidence retrieval and context length constraints. To address these challenges, we introduce a novel, retrieval-augmented LLMs framework--the first of its kind to automatically and strategically extract key evidence from web sources for claim verification. Employing a multi-round retrieval strategy, our framework ensures the acquisition of sufficient, relevant evidence, thereby enhancing performance. Comprehensive experiments across three real-world datasets validate the framework's superiority over existing methods. Importantly, our model not only delivers accurate verdicts but also offers human-readable explanations to improve result interpretability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑