arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1805.09653stat.MLcs.AIcs.LG
收藏

关注不确定性,实现可靠的解释和预测

Uncertainty-Aware Attention for Reliable Interpretation and Prediction

UNIST(蔚山科学技术院);KAIST(韩国科学技术院);AITrics(AITrics公司);Yonsei University College of Medicine(延世大学医学院);University of Oxford(牛津大学)

Jay Heo, Hae Beom Lee, Saehoon Kim, Juho Lee, Kwang Joon Kim, Eunho Yang, Sung Ju Hwang

摘要:

注意力机制在将深度学习模型聚焦于相关特征和解释它们方面都是有效的。然而,注意力可能是不可靠的,因为生成它们的网络通常是以弱监督的方式训练的。为了克服这一限制,我们将依赖于输入的不确定性的概念引入注意力机制,这样它就可以根据给定的输入为每个具有不同程度噪声的特征生成注意力,从而在不确定的情况下学习更大的方差。我们使用变分推断学习这种不确定性感知注意力(UA)机制,并在电子健康记录的各种风险预测任务上验证它,我们的模型显着优于现有的注意力模型。对学习注意力的分析表明,我们的模型产生的注意力符合临床医生的解释,并通过学习方差提供更丰富的解释。进一步评估的不确定度校准和预测性能的准确性与“我不知道”的决定表明,UA产量网络的可靠性也很高。

英文摘要:

Attention mechanism is effective in both focusing the deep learning models on relevant features and interpreting them. However, attentions may be unreliable since the networks that generate them are often trained in a weakly-supervised manner. To overcome this limitation, we introduce the notion of input-dependent uncertainty to the attention mechanism, such that it generates attention for each feature with varying degrees of noise based on the given input, to learn larger variance on instances it is uncertain about. We learn this Uncertainty-aware Attention (UA) mechanism using variational inference, and validate it on various risk prediction tasks from electronic health records on which our model significantly outperforms existing attention models. The analysis of the learned attentions shows that our model generates attentions that comply with clinicians' interpretation, and provide richer interpretation via learned variance. Further evaluation of both the accuracy of the uncertainty calibration and the prediction performance with "I don't know" decision show that UA yields networks with high reliability as well.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑