Ta'keed：首个面向阿拉伯语声明的生成式事实核查系统
Ta'keed: The First Generative Fact-Checking System for Arabic Claims
AI总结：
本文提出阿拉伯语可解释生成式事实核查系统 Ta'keed，结合信息检索与 LLM 验证，并构建 ArFactEx 数据集评估分类与解释生成效果。
摘要：
本文介绍了 Ta'keed，这是一个可解释的阿拉伯语自动事实核查系统。现有研究通常聚焦于将声明分类为“真”或“假”，但对生成有关声明可信度的解释探索有限，尤其是在阿拉伯语方面。Ta'keed 通过基于检索到的片段评估声明真实性来弥补这一空白，该系统采用两个主要组件：信息检索和基于 LLM 的声明验证。我们编制了 ArFactEx，这是一个带有手动论证参考资料的测试金标数据集，用于评估该系统。初始模型在分类任务中取得了 0.72 的可观 F1 分数。同时，系统生成的解释在句法和语义上与金标准解释进行了比较。研究建议使用语义相似度进行评估，所得平均余弦相似度得分为 0.76。此外，我们探究了不同片段数量对声明分类准确率的影响，揭示了潜在的相关性，其中使用前七个命中结果的模型表现优于其他模型，F1 得分为 0.77。
英文摘要：
This paper introduces Ta'keed, an explainable Arabic automatic fact-checking system. While existing research often focuses on classifying claims as "True" or "False," there is a limited exploration of generating explanations for claim credibility, particularly in Arabic. Ta'keed addresses this gap by assessing claim truthfulness based on retrieved snippets, utilizing two main components: information retrieval and LLM-based claim verification. We compiled the ArFactEx, a testing gold-labelled dataset with manually justified references, to evaluate the system. The initial model achieved a promising F1 score of 0.72 in the classification task. Meanwhile, the system's generated explanations are compared with gold-standard explanations syntactically and semantically. The study recommends evaluating using semantic similarities, resulting in an average cosine similarity score of 0.76. Additionally, we explored the impact of varying snippet quantities on claim classification accuracy, revealing a potential correlation, with the model using the top seven hits outperforming others with an F1 score of 0.77.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
