比较开放域科学声明验证的知识来源
Comparing Knowledge Sources for Open-Domain Scientific Claim Verification
AI总结：
本研究比较了开放域科学声明验证中不同知识来源与检索技术，发现PubMed适合专业生物医学声明，Wikipedia适合日常健康问题，BM25精度高而语义搜索召回率高。
摘要：
科学知识发现的速度日益加快，以及健康声明在网上的广泛传播，凸显了开发高效的科学声明事实核查系统的重要性。文献中此类任务的常规设置假设包含声明证据的文档已经提供并标注，或包含在有限的语料库中。这使得系统在现实世界场景中不切实际，因为在现实场景中，需要查询可能包含数百万文档的知识来源以找到相关证据。在本文中，我们进行了一系列实验来测试开放域声明验证系统的性能。我们在四个生物医学和健康声明数据集上，在不同设置下测试了系统的最终裁决预测。在保持流程中证据选择和裁决预测部分不变的情况下，文档检索通过三种常见知识来源（PubMed、Wikipedia、Google）并使用两种不同的信息检索技术进行。我们表明，PubMed在处理专业生物医学声明时效果更好，而Wikipedia更适合日常健康问题。同样，BM25在检索精度方面表现出色，而语义搜索在相关证据的召回率方面更优。我们讨论了结果，概述了常见的检索模式和挑战，并提供了有前景的未来方向。
英文摘要：
The increasing rate at which scientific knowledge is discovered and health claims shared online has highlighted the importance of developing efficient fact-checking systems for scientific claims. The usual setting for this task in the literature assumes that the documents containing the evidence for claims are already provided and annotated or contained in a limited corpus. This renders the systems unrealistic for real-world settings where knowledge sources with potentially millions of documents need to be queried to find relevant evidence. In this paper, we perform an array of experiments to test the performance of open-domain claim verification systems. We test the final verdict prediction of systems on four datasets of biomedical and health claims in different settings. While keeping the pipeline's evidence selection and verdict prediction parts constant, document retrieval is performed over three common knowledge sources (PubMed, Wikipedia, Google) and using two different information retrieval techniques. We show that PubMed works better with specialized biomedical claims, while Wikipedia is more suited for everyday health concerns. Likewise, BM25 excels in retrieval precision, while semantic search in recall of relevant evidence. We discuss the results, outline frequent retrieval patterns and challenges, and provide promising future directions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
