arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2010.12852cs.CVcs.AI
收藏

超越VQA:视觉问题的多词答案和理由生成

Beyond VQA: Generating Multi-word Answer and Rationale to Visual Questions

Indian Institute of Technology Hyderabad(印度理工学院海得拉巴分校)

Radhika Dua, Sai Srinivas Kancheti, Vineeth N Balasubramanian

摘要:

视觉问答是一个多模态任务,旨在测量高层次的视觉理解。当代VQA模型是限制性的,因为答案是通过对有限词汇的分类(在开放式VQA的情况下)或通过对一组多项选择类型答案的分类来获得的。在这项工作中,我们提出了一个完全生成的配方,多字的答案生成的视觉查询。为了更进一步,我们引入了一个新的任务:ViQAR(可视化问答和推理),其中模型必须生成完整的答案和试图证明生成的答案的理由。我们提出了一个端到端的架构来解决这个任务,并描述了如何评估它。我们表明,我们的模型产生强有力的答案和理由,通过定性和定量评估,以及通过人类图灵测试。

英文摘要:

Visual Question Answering is a multi-modal task that aims to measure high-level visual understanding. Contemporary VQA models are restrictive in the sense that answers are obtained via classification over a limited vocabulary (in the case of open-ended VQA), or via classification over a set of multiple-choice-type answers. In this work, we present a completely generative formulation where a multi-word answer is generated for a visual query. To take this a step forward, we introduce a new task: ViQAR (Visual Question Answering and Reasoning), wherein a model must generate the complete answer and a rationale that seeks to justify the generated answer. We propose an end-to-end architecture to solve this task and describe how to evaluate it. We show that our model generates strong answers and rationales through qualitative and quantitative evaluation, as well as through a human Turing Test.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑