arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2006.05419cs.LGcs.HCstat.ML
收藏

基于神经注意过程的高性价比交互式注意学习

Cost-effective Interactive Attention Learning with Neural Attention Processes

Korea Advanced Institute of Science and Technology (KAIST)(韩国科学技术院);Yonsei University College of Medicine(延世大学医学院);AITRICS

Jay Heo, Junhyeon Park, Hyewon Jeong, Kwang Joon Kim, Juho Lee, Eunho Yang, Sung Ju Hwang

摘要:

我们提出了一种新的交互式学习框架,我们称之为交互式注意力学习(IAL),其中人类主管交互式地操纵分配的注意力,通过更新注意力生成网络来纠正模型的行为。然而,由于缺乏人类注释,这样的模型容易过度拟合,并且需要昂贵的重新训练。此外,它几乎是不可行的,为人类的注释员,以审查对吨的实例和功能的关注。我们通过提出一种样本高效的注意力机制和一种针对实例和特征的经济高效的重新排序算法来应对这些挑战。首先,我们提出了神经注意力过程(NAP),这是一个注意力发生器,可以通过合并新的注意力水平监督来更新其行为,而无需任何重新训练。其次,我们提出了一种算法,优先考虑的实例和功能的负面影响,使模型可以产生很大的改善与最小的人的反馈。我们在来自多个领域(医疗保健,房地产和计算机视觉)的各种时间序列数据集上验证了IAL,在这些数据集上,IAL的性能显着优于传统注意力机制的基线,或者没有成本效益的重新排序,具有更少的再培训和人类模型交互成本。

英文摘要:

We propose a novel interactive learning framework which we refer to as Interactive Attention Learning (IAL), in which the human supervisors interactively manipulate the allocated attentions, to correct the model's behavior by updating the attention-generating network. However, such a model is prone to overfitting due to scarcity of human annotations, and requires costly retraining. Moreover, it is almost infeasible for the human annotators to examine attentions on tons of instances and features. We tackle these challenges by proposing a sample-efficient attention mechanism and a cost-effective reranking algorithm for instances and features. First, we propose Neural Attention Process (NAP), which is an attention generator that can update its behavior by incorporating new attention-level supervisions without any retraining. Secondly, we propose an algorithm which prioritizes the instances and the features by their negative impacts, such that the model can yield large improvements with minimal human feedback. We validate IAL on various time-series datasets from multiple domains (healthcare, real-estate, and computer vision) on which it significantly outperforms baselines with conventional attention mechanisms, or without cost-effective reranking, with substantially less retraining and human-model interaction cost.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑