arXivDaily arXiv每日学术速递 周一至周五更新
看起来今天可以划水了
arXiv 2207.09899cs.CV
收藏

标记指令在生物医学图像分析中很重要

Labeling instructions matter in biomedical image analysis

German Cancer Research Center (DKFZ)(德国癌症研究中心);Heidelberg University(海德堡大学);Helmholtz Zentrum München(亥姆霍兹慕尼黑中心);Quality Match GmbH(Quality Match有限责任公司)

Tim Rädsch, Annika Reinke, Vivienn Weru, Minu D. Tizabi, Nicholas Schreck, A. Emre Kavur, Bünyamin Pekdemir, Tobias Roß, Annette Kopp-Schneider, Lena Maier-Hein

摘要:

生物医学图像分析算法验证依赖于参考数据集的高质量注释,其中标记说明是关键。尽管它们很重要,但它们的优化在很大程度上仍未得到探索。在这里,我们提出了第一个系统的研究,标签的指示和他们的影响,注释质量的领域。通过对专业实践和在MICCAI协会注册的国际比赛的全面检查,我们发现了注释者对标签说明的需求与其当前质量和可用性之间的差异。基于对来自四家专业公司的156名注释者和708名Amazon Mechanical Turk(MTurk)群众工作者使用不同信息密度级别的指令注释的14,040张图像的分析,我们进一步发现,与纯文本描述相比,包含示例性图像显着提高了注释性能,而仅仅扩展文本描述则没有。最后,专业注释者的表现一直优于MTurk众包工作者。我们的研究提高了人们对生物医学图像分析标签说明质量标准需求的认识。

英文摘要:

Biomedical image analysis algorithm validation depends on high-quality annotation of reference datasets, for which labeling instructions are key. Despite their importance, their optimization remains largely unexplored. Here, we present the first systematic study of labeling instructions and their impact on annotation quality in the field. Through comprehensive examination of professional practice and international competitions registered at the MICCAI Society, we uncovered a discrepancy between annotators' needs for labeling instructions and their current quality and availability. Based on an analysis of 14,040 images annotated by 156 annotators from four professional companies and 708 Amazon Mechanical Turk (MTurk) crowdworkers using instructions with different information density levels, we further found that including exemplary images significantly boosts annotation performance compared to text-only descriptions, while solely extending text descriptions does not. Finally, professional annotators constantly outperform MTurk crowdworkers. Our study raises awareness for the need of quality standards in biomedical image analysis labeling instructions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑