arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2007.05351cs.CVcs.LGeess.IVq-bio.QM
收藏

病理学家定义的标签是否可重复?TUPAC16有丝分裂像数据集与替代标签集的比较

Are pathologist-defined labels reproducible? Comparison of the TUPAC16 mitotic figure dataset with an alternative set of labels

Freie Universität Berlin(柏林自由大学);Eindhoven University of Technology(埃因霍温理工大学);Friedrich-Alexander-Universität Erlangen-Nürnberg(埃尔朗根-纽伦堡弗里德里希-亚历山大大学);University Medical Center Utrecht(乌得勒支大学医学中心)

Christof A. Bertram, Mitko Veta, Christian Marzahl, Nikolas Stathonikos, Andreas Maier, Robert Klopfleisch, Marc Aubreville

摘要:

病理学家定义的标签是组织病理学数据集的金标准,而不考虑某些任务在一致性方面的众所周知的限制。迄今为止,已经有一些关于有丝分裂像的数据集可用,并用于开发有前途的基于深度学习的算法。为了评估这些算法的鲁棒性和方法的可重复性,有必要在几个独立的数据集上进行测试。这些可用数据集的不同标记方法的影响目前尚不清楚。为了解决这个问题,我们为TUPAC 16挑战的辅助有丝分裂数据集的图像提供了一组替代标签。除了手动有丝分裂像筛选外,我们还使用了一种新的算法辅助标记过程,可以最大限度地降低图像中缺失罕见有丝分裂像的风险。所有潜在的有丝分裂像由两名病理学家独立评估。新的公开可用的标签集包含1,999个有丝分裂图(+28.80%),另外还包括10,483个与有丝分裂图高度相似的细胞标签(硬例子)。我们发现使用标准深度学习目标检测架构比较原始标签集(0.549)和新的替代标签集(0.735)之间的F_1分数存在显著差异。在替代集上训练的模型显示出更高的总体置信度值,表明总体标签一致性更高。本研究的结果表明,病理学家定义的标签可能会有显着差异,导致模型性能的显着差异。在具有不同标记方法的独立数据集之间比较基于深度学习的算法应谨慎进行。

英文摘要:

Pathologist-defined labels are the gold standard for histopathological data sets, regardless of well-known limitations in consistency for some tasks. To date, some datasets on mitotic figures are available and were used for development of promising deep learning-based algorithms. In order to assess robustness of those algorithms and reproducibility of their methods it is necessary to test on several independent datasets. The influence of different labeling methods of these available datasets is currently unknown. To tackle this, we present an alternative set of labels for the images of the auxiliary mitosis dataset of the TUPAC16 challenge. Additional to manual mitotic figure screening, we used a novel, algorithm-aided labeling process, that allowed to minimize the risk of missing rare mitotic figures in the images. All potential mitotic figures were independently assessed by two pathologists. The novel, publicly available set of labels contains 1,999 mitotic figures (+28.80%) and additionally includes 10,483 labels of cells with high similarities to mitotic figures (hard examples). We found significant difference comparing F_1 scores between the original label set (0.549) and the new alternative label set (0.735) using a standard deep learning object detection architecture. The models trained on the alternative set showed higher overall confidence values, suggesting a higher overall label consistency. Findings of the present study show that pathologists-defined labels may vary significantly resulting in notable difference in the model performance. Comparison of deep learning-based algorithms between independent datasets with different labeling methods should be done with caution.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑