基于注意记忆的视觉对话参考消解
Visual Reference Resolution using Attention Memory for Visual Dialog
摘要：
视觉对话是一种在输入图像的情况下回答一系列相互依赖的问题的任务，并且通常需要解决问题之间的视觉参考。这个问题与视觉问答（VQA）不同，视觉问答依赖于空间注意力（又名视觉基础）从图像和问题对估计。我们提出了一种新的注意力机制，利用过去的视觉注意力来解决视觉对话场景中的当前参考。该模型配备了一个关联的注意力记忆存储序列的先前（注意，关键）对。从这个记忆中，模型检索先前的注意力，考虑到与当前问题最相关的新近度，以解决潜在的模糊引用。然后，该模型将检索到的注意力与试探性的注意力合并，以获得当前问题的最终注意力;具体地说，我们使用动态参数预测来结合以问题为条件的两个注意力。通过对一个新的合成视觉对话数据集进行广泛的实验，我们表明，我们的模型在视觉参考分辨率起重要作用的情况下显着优于最先进的模型（约16%）。此外，所提出的模型在Visual Dialog数据集中实现了卓越的性能（约2%的点改进），尽管参数比基线少得多。
英文摘要：
Visual dialog is a task of answering a series of inter-dependent questions given an input image, and often requires to resolve visual references among the questions. This problem is different from visual question answering (VQA), which relies on spatial attention (a.k.a. visual grounding) estimated from an image and question pair. We propose a novel attention mechanism that exploits visual attentions in the past to resolve the current reference in the visual dialog scenario. The proposed model is equipped with an associative attention memory storing a sequence of previous (attention, key) pairs. From this memory, the model retrieves the previous attention, taking into account recency, which is most relevant for the current question, in order to resolve potentially ambiguous references. The model then merges the retrieved attention with a tentative one to obtain the final attention for the current question; specifically, we use dynamic parameter prediction to combine the two attentions conditioned on the question. Through extensive experiments on a new synthetic visual dialog dataset, we show that our model significantly outperforms the state-of-the-art (by ~16 % points) in situations, where visual reference resolution plays an important role. Moreover, the proposed model achieves superior performance (~ 2 % points improvement) in the Visual Dialog dataset, despite having significantly fewer parameters than the baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
