基于RAG与大语言模型少样本上下文学习的证据支撑式事实核查
Evidence-backed Fact Checking using RAG and Few-Shot In-Context Learning with LLMs
AI总结：
针对社交媒体错误信息泛滥、人工核查困难的问题，提出结合RAG流水线与LLM少样本上下文学习的自动化事实核查系统，可输出真实性判断及支撑证据，在Averitec数据集上较基线得分提升22%。
摘要：
鉴于错误信息在社交媒体上广泛传播，为在线声明建立事实核查机制至关重要。人工核实每一条声明难度极大，凸显了自动化事实核查系统的必要性。本文提出了旨在解决这一问题的系统。我们使用Averitec数据集（Schlichtkrull等人，2023）评估所提事实核查系统的性能。除真实性预测外，系统还能提供从数据集中提取的支撑证据。我们构建了Retrieve and Generate（RAG，检索生成）流水线，从知识库中提取相关证据句，再将其与声明一同输入大语言模型（LLM）进行分类。我们还评估了多个LLM的少样本In-Context Learning（ICL，上下文学习）能力。该系统的“Averitec”得分达到0.33，较基线绝对提升22%。我们的代码已公开于https://github.com/ronit-singhal/evidence-backed-fact-checking-using-rag-and-few-shot-in-context-learning-with-llms。
英文摘要：
Given the widespread dissemination of misinformation on social media, implementing fact-checking mechanisms for online claims is essential. Manually verifying every claim is very challenging, underscoring the need for an automated fact-checking system. This paper presents our system designed to address this issue. We utilize the Averitec dataset (Schlichtkrull et al., 2023) to assess the performance of our fact-checking system. In addition to veracity prediction, our system provides supporting evidence, which is extracted from the dataset. We develop a Retrieve and Generate (RAG) pipeline to extract relevant evidence sentences from a knowledge base, which are then inputted along with the claim into a large language model (LLM) for classification. We also evaluate the few-shot In-Context Learning (ICL) capabilities of multiple LLMs. Our system achieves an 'Averitec' score of 0.33, which is a 22% absolute improvement over the baseline. Our Code is publicly available on https://github.com/ronit-singhal/evidence-backed-fact-checking-using-rag-and-few-shot-in-context-learning-with-llms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
