生物医学系统评价的众包信息抽取
Crowdsourcing Information Extraction for Biomedical Systematic Reviews
摘要：
信息抽取是进行生物医学系统文献综述实践中的关键步骤。提取的结构化数据可以通过统计元分析等方法进行汇总。通常，训练有素的领域专家提取数据进行系统评价。进行生物医学系统评价的高昂费用促使研究人员探索成本较低的方法，以实现类似的严格性，而不影响质量。众包就是这样一种有前途的办法。在这项正在进行的研究中，我们设计了一个众包的生物医学信息抽取的任务。我们简要报告迭代设计过程和两个试点测试的结果。我们发现，在任务说明中给出更具体的例子可以帮助工人更好地理解任务，特别是对于抽象和令人困惑的概念。我们发现，只有少数工人完成了大部分工作，而且我们的薪酬水平似乎对低收入国家的工人更具吸引力。在未来，我们将进一步评估我们的结果与参考金标准提取，从而评估任务人群工作者提取生物医学干预信息进行系统评价的可行性。
英文摘要：
Information extraction is a critical step in the practice of conducting biomedical systematic literature reviews. Extracted structured data can be aggregated via methods such as statistical meta-analysis. Typically highly trained domain experts extract data for systematic reviews. The high expense of conducting biomedical systematic reviews has motivated researchers to explore lower cost methods that achieve similar rigor without compromising quality. Crowdsourcing represents one such promising approach. In this work-in-progress study, we designed a crowdsourcing task for biomedical information extraction. We briefly report the iterative design process and the results of two pilot testings. We found that giving more concrete examples in the task instruction can help workers better understand the task, especially for concepts that are abstract and confusing. We found a few workers completed most of the work, and our payment level appeared more attractive to workers from low-income countries. In the future, we will further evaluate our results with reference to gold standard extractions, thus assessing the feasibility of tasking crowd workers with extracting biomedical intervention information for systematic reviews.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
