大语言模型能否提取出具有人类细粒度证据的证据以用于基于证据的事实核查？
Can LLMs extract human-like fine-grained evidence for evidence-based fact-checking?
AI总结：
本文研究了大语言模型在细粒度证据提取任务中的表现，通过新数据集评估发现，模型大小与对齐程度之间存在有效平衡。
摘要：
在线新闻文章中的用户评论中经常出现虚假信息，这突显了需要有效方法来检测事实性错误信息的必要性。为了强烈支持或反驳从这些评论中提取的声明，有必要识别相关文档并确定精确的文本片段来证明或反驳每个声明。本文聚焦于后者任务——为捷克语和斯洛伐克语声明进行细粒度证据提取。我们创建了一个新数据集，其中包含由付费标注员创建的双向标注细粒度证据。我们在此数据集上评估大型语言模型（LLMs），以评估其与人类标注的一致性。结果表明，LLMs经常无法直接从源文本中复制证据，导致无效输出。错误率分析显示，llama3.1:8b模型尽管相对较小，却实现了较高的正确输出比例，而gpt-oss-120b模型尽管参数更多，却表现不佳。此外，qwen3:14b、deepseek-r1:32b和gpt-oss:20b模型在模型大小与与人类标注的一致性之间表现出有效的平衡。
英文摘要：
Misinformation frequently spreads in user comments under online news articles, highlighting the need for effective methods to detect factually incorrect information. To strongly support or refute claims extracted from such comments, it is necessary to identify relevant documents and pinpoint the exact text spans that justify or contradict each claim. This paper focuses on the latter task -- fine-grained evidence extraction for Czech and Slovak claims. We create new dataset, containing two-way annotated fine-grained evidence created by paid annotators. We evaluate large language models (LLMs) on this dataset to assess their alignment with human annotations. The results reveal that LLMs often fail to copy evidence verbatim from the source text, leading to invalid outputs. Error-rate analysis shows that the {llama3.1:8b model achieves a high proportion of correct outputs despite its relatively small size, while the gpt-oss-120b model underperforms despite having many more parameters. Furthermore, the models qwen3:14b, deepseek-r1:32b, and gpt-oss:20b demonstrate an effective balance between model size and alignment with human annotations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
