可视化对话框的概率求解框架
Probabilistic framework for solving Visual Dialog
摘要:
在本文中,我们提出了一个概率框架,解决任务的“可视对话框”。解决这个问题需要推理和理解的视觉形态,语言形态和常识知识来回答。已经提出了各种架构来通过组合视觉和语言表示的多模态深度学习技术的变体来解决这个任务。然而,我们认为,理解和分析不确定性的来源对于解决这一任务至关重要。我们的方法允许估计不确定性,也有助于产生不同的答案。所提出的方法是通过一个概率表示模块,为我们提供了图像,问题和会话历史的表示,一个模块,确保候选答案的不同的潜在表示得到给定的概率表示和不确定性表示模块,选择适当的答案,最大限度地减少不确定性。我们通过详细的消融分析、与最先进技术的比较以及有助于理解该方法的不确定性的可视化来彻底评估该模型。使用所提出的概率框架,我们从而获得一个改进的视觉对话系统,也更易于解释。
英文摘要:
In this paper, we propose a probabilistic framework for solving the task of `Visual Dialog'. Solving this task requires reasoning and understanding of visual modality, language modality, and common sense knowledge to answer. Various architectures have been proposed to solve this task by variants of multi-modal deep learning techniques that combine visual and language representations. However, we believe that it is crucial to understand and analyze the sources of uncertainty for solving this task. Our approach allows for estimating uncertainty and also aids a diverse generation of answers. The proposed approach is obtained through a probabilistic representation module that provides us with representations for image, question and conversation history, a module that ensures that diverse latent representations for candidate answers are obtained given the probabilistic representations and an uncertainty representation module that chooses the appropriate answer that minimizes uncertainty. We thoroughly evaluate the model with a detailed ablation analysis, comparison with state of the art and visualization of the uncertainty that aids in the understanding of the method. Using the proposed probabilistic framework, we thus obtain an improved visual dialog system that is also more explainable.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。