arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2009.05094cs.LG
收藏

为什么我不放弃:了解癌症病理学报告弃权分类器的分类决定因素

Why I'm not Answering: Understanding Determinants of Classification of an Abstaining Classifier for Cancer Pathology Reports

Los Alamos National Laboratory(洛斯阿拉莫斯国家实验室);Louisiana Tumor Registry(路易斯安那州肿瘤登记处);Kentucky Cancer Registry(肯塔基州癌症登记处);Utah Cancer Registry(犹他州癌症登记处);New Jersey State Cancer Registry(新泽西州癌症登记处);Oak Ridge National Laboratory(橡树岭国家实验室);Information Management Services Inc.(信息管理服务公司);National Cancer Institute(美国国家癌症研究所)
Sayera Dhaubhadel, Jamaludin Mohd-Yusof, Kumkum Ganguly, Gopinath Chennupati, Sunil Thulasidasan, Nicolas W. Hengartner, Brent J. Mumphrey, Eric B. Durbin, Jenn… 展开作者

Sayera Dhaubhadel, Jamaludin Mohd-Yusof, Kumkum Ganguly, Gopinath Chennupati, Sunil Thulasidasan, Nicolas W. Hengartner, Brent J. Mumphrey, Eric B. Durbin, Jennifer A. Doherty, Mireille Lemieux, Noah Schaefferkoetter, Georgia Tourassi, Linda Coyle, Lynne Penberthy, Benjamin H. McMahon, Tanmoy Bhattacharya

摘要:

在关键的现实世界应用中安全部署深度学习系统要求模型只在可预测的情况下犯很少的错误。在这项工作中,我们解决这个问题,使用一个弃权分类器,调整到有$>$ 95%的准确性,然后确定使用石灰弃权的决定因素。从本质上讲,我们正在训练我们的模型来学习病理报告的属性,这些属性可能导致不正确的分类,尽管代价是灵敏度降低。我们展示了一个弃权分类器在多任务设置分类癌症病理报告从NCI SEER癌症登记册上的六个任务的兴趣。对于这些任务,我们通过对2--5 - 45%的报告弃权,将分类出错率降低2- 5倍。对于癌症部位分类的具体任务,我们能够识别转移、涉及淋巴结的报告以及对多个癌症部位的讨论,作为许多分类错误的原因,并且观察到错误的程度和类型随癌症部位系统地变化(例如,乳腺、肺和前列腺)。当结合三个任务时,我们的模型对50%的报告进行分类,其中三个任务\edit的准确率大于95%,而保留样本的所有六个任务的准确率大于85%。此外,我们表明,石灰提供了一个更好的决定因素的分类比单独的字出现的措施。通过将深度弃权分类器与使用LIME的特征识别相结合,我们能够在从病理报告中对癌症部位进行分类时识别负责正确性和预防性的概念。LIME相对于关键词搜索的改进在统计上是显著的,大概是因为词是在上下文中评估的,并且被确定为分类的局部决定因素。

英文摘要:

Safe deployment of deep learning systems in critical real world applications requires models to make very few mistakes, and only under predictable circumstances. In this work, we address this problem using an abstaining classifier that is tuned to have $>$95% accuracy, and then identify the determinants of abstention using LIME. Essentially, we are training our model to learn the attributes of pathology reports that are likely to lead to incorrect classifications, albeit at the cost of reduced sensitivity. We demonstrate an abstaining classifier in a multitask setting for classifying cancer pathology reports from the NCI SEER cancer registries on six tasks of interest. For these tasks, we reduce the classification error rate by factors of 2--5 by abstaining on 25--45% of the reports. For the specific task of classifying cancer site, we are able to identify metastasis, reports involving lymph nodes, and discussion of multiple cancer sites as responsible for many of the classification mistakes, and observe that the extent and types of mistakes vary systematically with cancer site (e.g., breast, lung, and prostate). When combining across three of the tasks, our model classifies 50% of the reports with an accuracy greater than 95% for three of the six tasks\edit, and greater than 85% for all six tasks on the retained samples. Furthermore, we show that LIME provides a better determinant of classification than measures of word occurrence alone. By combining a deep abstaining classifier with feature identification using LIME, we are able to identify concepts responsible for both correctness and abstention when classifying cancer sites from pathology reports. The improvement of LIME over keyword searches is statistically significant, presumably because words are assessed in context and have been identified as a local determinant of classification.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑