基于LLM的众包事实核查中证据摘要的效率与有效性
Efficiency and Effectiveness of LLM-Based Summarization of Evidence in Crowdsourced Fact-Checking
AI总结：
本研究通过A/B测试对比了完整网页与LLM生成的摘要作为证据在众包事实核查中的表现，发现摘要证据在保持相当准确率的同时显著提升了效率、降低了成本并提高了内部一致性。
摘要：
评估在线内容的真实性对于打击虚假信息至关重要。本研究通过比较两种方法，考察了众包真实性评估的效率与有效性：一种方法以每个声明的完整网页作为证据，另一种方法使用由大型语言模型生成的每个证据文档的摘要。在A/B测试设置下，我们招募了多样化的参与者群体，要求他们在这些条件下评估声明的真实性。我们的分析探讨了评估质量和参与者的行为模式。结果显示，依赖摘要证据在准确率和误差指标上与标准模式相当，同时显著提升了效率。摘要设置下的工作者完成了数量显著更多的评估，降低了任务耗时和成本。此外，摘要模式最大化了内部一致性，并保持了对证据一致的依赖度和感知有用性，证明了其在简化大规模真实性评估方面的潜力。
英文摘要：
Evaluating the truthfulness of online content is critical for combating misinformation. This study examines the efficiency and effectiveness of crowdsourced truthfulness assessments through a comparative analysis of two approaches: one involving full-length webpages as evidence for each claim, and another using summaries for each evidence document generated with a large language model. Using an A/B testing setting, we engage a diverse pool of participants tasked with evaluating the truthfulness of statements under these conditions. Our analysis explores both the quality of assessments and the behavioral patterns of participants. The results reveal that relying on summarized evidence offers comparable accuracy and error metrics to the Standard modality while significantly improving efficiency. Workers in the Summary setting complete a significantly higher number of assessments, reducing task duration and costs. Additionally, the Summary modality maximizes internal agreement and maintains consistent reliance on and perceived usefulness of evidence, demonstrating its potential to streamline large-scale truthfulness evaluations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
