Attention mechanism is effective in both focusing the deep learning models on relevant features and interpreting them. However, attentions may be unreliable since the networks that generate them are often trained in a weakly-supervised manner. To overcome this limitation, we introduce the notion of input-dependent uncertainty to the attention mechanism, such that it generates attention for each feature with varying degrees of noise based on the given input, to learn larger variance on instances it is uncertain about. We learn this Uncertainty-aware Attention (UA) mechanism using variational inference, and validate it on various risk prediction tasks from electronic health records on which our model significantly outperforms existing attention models. The analysis of the learned attentions shows that our model generates attentions that comply with clinicians' interpretation, and provide richer interpretation via learned variance. Further evaluation of both the accuracy of the uncertainty calibration and the prediction performance with "I don't know" decision show that UA yields networks with high reliability as well.
Incremental Uncertainty-aware Performance Monitoring with Active Labeling Intervention
Siemens AG(西门子公司);Goethe University Frankfurt(法兰克福歌德大学);LMU Munich(慕尼黑大学);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);German Cancer Research Center (DKFZ)(德国癌症研究中心);German Cancer Consortium (DKTK)(德国癌症联盟)
Uncertainty-Aware Time-to-Event Prediction using Deep Kernel Accelerated Failure Time Models
Ludwig Maximilians University Munich(慕尼黑路德维希-马克西米利安大学);Siemens AG(西门子股份公司);University Hospital Erlangen(埃尔朗根大学医院)
基于递归神经网络的解决方案越来越多地用于纵向电子健康记录数据的分析。然而,大多数工作都集中在预测准确性上,而忽视了预测的不确定性。我们提出了用于时间到事件预测任务的深度内核加速故障时间模型,通过递归神经网络和稀疏高斯过程的管道实现预测的不确定性意识。此外,基于深度度量学习的预训练步骤适于增强所提出的模型。在两个真实数据集上的实验中,我们的模型显示出比基于递归神经网络的基线更好的点估计性能。更重要的是,我们模型的预测方差可用于量化事件发生时间预测的不确定性估计:当我们的模型对其预测更有信心时,它会提供更好的性能。与相关方法(如Monte Carlo Dropout)相比,我们的模型通过利用分析解决方案提供了更好的不确定性估计,并且计算效率更高。