arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1709.05168cs.IRcs.HC
收藏

系统性文献综述中的众包论文筛选

Crowdsourcing Paper Screening in Systematic Literature Reviews

Evgeny Krivosheev, Fabio Casati, Valentina Caforio, Boualem Benatallah

摘要:

文献综述让科学家站在巨人的肩膀上,展示有希望的方向,总结进展,指出研究中存在的挑战。与此同时,进行系统的文献综述是一个费力和昂贵的过程。在过去的十年中,文献综述中对众包的研究很少。本文探讨了众包的可行性,以促进文献综述过程中的结果,时间和精力,以及确定众包策略提供最佳的结果的基础上,可用的预算。特别是,我们专注于文献综述过程的筛选阶段,我们贡献和评估用于确定测试大小的方法,每篇论文所需的标签和分类功能,以及将众包过程分阶段进行的方法,以改善结果。最后,我们提出了我们的研究结果的基础上运行的实验Crowdflower。

英文摘要:

Literature reviews allow scientists to stand on the shoulders of giants, showing promising directions, summarizing progress, and pointing out existing challenges in research. At the same time conducting a systematic literature review is a laborious and consequently expensive process. In the last decade, there have a few studies on crowdsourcing in literature reviews. This paper explores the feasibility of crowdsourcing for facilitating the literature review process in terms of results, time and effort, as well as to identify which crowdsourcing strategies provide the best results based on the budget available. In particular we focus on the screening phase of the literature review process and we contribute and assess methods for identifying the size of tests, labels required per paper, and classification functions as well as methods to split the crowdsourcing process in phases to improve results. Finally, we present our findings based on experiments run on Crowdflower.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑