使用检索增强型大语言模型代理进行长期形式的新冠事实核查
Use of Retrieval-Augmented Large Language Model Agent for Long-Form COVID-19 Fact-Checking
AI总结：
本文提出SAFE系统，结合大语言模型与检索增强生成技术，有效提升长期新冠虚假信息的事实核查能力。
摘要：
新冠信息泛滥呼吁可扩展的事实核查解决方案，能够准确可靠地处理长篇虚假信息。本研究提出了SAFE（准确事实提取和评估系统），一种结合大语言模型与检索增强生成（RAG）的代理系统，以提高对长期新冠虚假信息的自动化事实核查。SAFE包括两个代理：一个用于声明提取，另一个使用LOTR-RAG进行声明验证，后者利用包含130,000篇新冠研究文献的文献库。增强变体SAFE（LOTR-RAG + SRAG）结合了Self-RAG，通过查询重写优化检索。我们在50篇假新闻文章（2-17页）上评估了这两个系统，这些文章包含246个标注声明（均值=4.922，标准差=3.186），由公共卫生专家标记为真实（14.1%）、部分真实（14.4%）、虚假（27.0%）、部分虚假（2.2%）和误导性（21.0%）。SAFE系统在所有指标上均显著优于基线LLM（p < 0.001）。在一致性（0-1尺度）方面，SAFE（LOTR-RAG）得分为0.629，优于SAFE（+SRAG）（0.577）和基线（0.279）。在主观评估（0-4利克特尺度）中，SAFE（LOTR-RAG）在有用性（3.640）、清晰性（3.800）和真实性（3.526）方面也取得了最高平均评分。添加SRAG略微降低了整体性能，除了在清晰性上略有提升。SAFE通过解决LLM在一致性和可解释性方面的局限性，在长期新冠事实核查中展示了稳健的改进。核心的LOTR-RAG设计比其SRAG增强变体更有效，为可扩展的虚假信息缓解提供了坚实的基础。
英文摘要：
The COVID-19 infodemic calls for scalable fact-checking solutions that handle long-form misinformation with accuracy and reliability. This study presents SAFE (system for accurate fact extraction and evaluation), an agent system that combines large language models with retrieval-augmented generation (RAG) to improve automated fact-checking of long-form COVID-19 misinformation. SAFE includes two agents - one for claim extraction and another for claim verification using LOTR-RAG, which leverages a 130,000-document COVID-19 research corpus. An enhanced variant, SAFE (LOTR-RAG + SRAG), incorporates Self-RAG to refine retrieval via query rewriting. We evaluated both systems on 50 fake news articles (2-17 pages) containing 246 annotated claims (M = 4.922, SD = 3.186), labeled as true (14.1%), partly true (14.4%), false (27.0%), partly false (2.2%), and misleading (21.0%) by public health professionals. SAFE systems significantly outperformed baseline LLMs in all metrics (p < 0.001). For consistency (0-1 scale), SAFE (LOTR-RAG) scored 0.629, exceeding both SAFE (+SRAG) (0.577) and the baseline (0.279). In subjective evaluations (0-4 Likert scale), SAFE (LOTR-RAG) also achieved the highest average ratings in usefulness (3.640), clearness (3.800), and authenticity (3.526). Adding SRAG slightly reduced overall performance, except for a minor gain in clearness. SAFE demonstrates robust improvements in long-form COVID-19 fact-checking by addressing LLM limitations in consistency and explainability. The core LOTR-RAG design proved more effective than its SRAG-augmented variant, offering a strong foundation for scalable misinformation mitigation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
