arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2012.02495cs.CV
收藏

How Many Annotators Do We Need? -- A Study on the Influence of Inter-Observer Variability on the Reliability of Automatic Mitotic Figure Assessment

Friedrich-Alexander-Universität Erlangen-Nürnberg(埃尔朗根-纽伦堡弗里德里希-亚历山大大学);Freie Universität Berlin(柏林自由大学);Animal Medical Center(动物医疗中心);University of Pennsylvania(宾夕法尼亚大学);University of Veterinary Medicine Hannover(汉诺威兽医大学);Synlab’s VPG Histology(Synlab VPG组织学部门);Michigan State University(密歇根州立大学);Idorsia Pharmaceuticals Ltd(Idorsia制药有限公司);International Zoo Veterinary Group(国际动物园兽医集团);University of Bern(伯尔尼大学);IDEXX Vet Med Labor GmbH(爱德士兽医实验室有限责任公司)
Frauke Wilm, Christof A. Bertram, Christian Marzahl, Alexander Bartel, Taryn A. Donovan, Charles-Antoine Assenmacher, Kathrin Becker, Mark Bennett, Sarah Corner… 展开作者

Frauke Wilm, Christof A. Bertram, Christian Marzahl, Alexander Bartel, Taryn A. Donovan, Charles-Antoine Assenmacher, Kathrin Becker, Mark Bennett, Sarah Corner, Brieuc Cossic, Daniela Denk, Martina Dettwiler, Beatriz Garcia Gonzalez, Corinne Gurtner, Annika Lehmbecker, Sophie Merz, Stephanie Plog, Anja Schmidt, Rebecca C. Smedley, Marco Tecilla, Tuddow Thaiwong, Katharina Breininger, Matti Kiupel, Andreas Maier, Robert Klopfleisch, Marc Aubreville

英文摘要:

Density of mitotic figures in histologic sections is a prognostically relevant characteristic for many tumours. Due to high inter-pathologist variability, deep learning-based algorithms are a promising solution to improve tumour prognostication. Pathologists are the gold standard for database development, however, labelling errors may hamper development of accurate algorithms. In the present work we evaluated the benefit of multi-expert consensus (n = 3, 5, 7, 9, 11) on algorithmic performance. While training with individual databases resulted in highly variable F$_1$ scores, performance was notably increased and more consistent when using the consensus of three annotators. Adding more annotators only resulted in minor improvements. We conclude that databases by few pathologists and high label accuracy may be the best compromise between high algorithmic performance and time investment.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑