arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2007.05611cs.LGcs.AIstat.ML
收藏

使用反蒸馏进行深度背景临床预测

Deep Contextual Clinical Prediction with Reverse Distillation

MIT CSAIL & IMES(麻省理工学院计算机科学与人工智能实验室及医学工程与科学研究所);Independence Blue Cross(独立蓝十字保险公司)

Rohan S. Kodialam, Rebecca Boiarsky, Justin Lim, Neil Dixit, Aditya Sai, David Sontag

摘要:

医疗保健提供者越来越多地使用机器学习来预测患者的结果,以进行有意义的干预。然而,尽管在这一领域有创新,但深度学习模型在预测这些结果时往往难以与浅层线性模型的性能相匹配,因此很难在实践中利用这些技术。在这项工作中,受保险索赔临床预测任务的激励,我们提出了一种名为反向蒸馏的新技术,该技术通过使用高性能线性模型进行初始化来预训练深度模型。我们利用保险索赔数据集的纵向结构来开发具有反向蒸馏的自注意力,或SARD,这是一种结合了上下文嵌入,时间嵌入和自我注意力机制的架构,最重要的是通过反向蒸馏进行训练。SARD在多个临床预测结果方面优于最先进的方法,消融研究表明,逆向蒸馏是这些改善的主要驱动力。代码可在https://github.com/clinicalml/omop-learn上获得。

英文摘要:

Healthcare providers are increasingly using machine learning to predict patient outcomes to make meaningful interventions. However, despite innovations in this area, deep learning models often struggle to match performance of shallow linear models in predicting these outcomes, making it difficult to leverage such techniques in practice. In this work, motivated by the task of clinical prediction from insurance claims, we present a new technique called Reverse Distillation which pretrains deep models by using high-performing linear models for initialization. We make use of the longitudinal structure of insurance claims datasets to develop Self Attention with Reverse Distillation, or SARD, an architecture that utilizes a combination of contextual embedding, temporal embedding and self-attention mechanisms and most critically is trained via reverse distillation. SARD outperforms state-of-the-art methods on multiple clinical prediction outcomes, with ablation studies revealing that reverse distillation is a primary driver of these improvements. Code is available at https://github.com/clinicalml/omop-learn.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑