arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2204.00879cs.CL
收藏

Co-VQA:通过交互式子问题序列进行提问

Co-VQA : Answering by Interactive Sub Question Sequence

Beijing University of Posts and Telecommunications(北京邮电大学);Meituan-Dianping Group(美团点评集团)

Ruonan Wang, Yuxi Qian, Fangxiang Feng, Xiaojie Wang, Huixing Jiang

摘要:

现有的可视化问答(VQA)方法大多是直接回答问题,但人们通常将复杂问题分解为一系列简单的子问题,然后通过回答子问题序列(SQS)来获得原问题的答案。通过模拟这一过程,本文提出了一个基于会话的VQA(Co-VQA)框架,该框架由三个组件组成:提问者、Oracle和回答者。Questioner使用扩展的HRED模型提出子问题,Oracle逐一回答。提出了一种自适应链式视觉推理模型(ACVRM),该模型利用问答对来更新视觉表征。为了对每个模型进行监督学习,我们引入了一种精心设计的方法,在VQA 2.0和VQA-CP v2数据集上为每个问题构建SQS。实验结果表明,该方法在VQA-CP v2上达到了最佳效果。进一步的分析表明,SQS有助于在问题和图像之间建立直接的语义联系,提供问题自适应的变长推理链,并具有明确的可解释性和错误可追溯性。

英文摘要:

Most existing approaches to Visual Question Answering (VQA) answer questions directly, however, people usually decompose a complex question into a sequence of simple sub questions and finally obtain the answer to the original question after answering the sub question sequence(SQS). By simulating the process, this paper proposes a conversation-based VQA (Co-VQA) framework, which consists of three components: Questioner, Oracle, and Answerer. Questioner raises the sub questions using an extending HRED model, and Oracle answers them one-by-one. An Adaptive Chain Visual Reasoning Model (ACVRM) for Answerer is also proposed, where the question-answer pair is used to update the visual representation sequentially. To perform supervised learning for each model, we introduce a well-designed method to build a SQS for each question on VQA 2.0 and VQA-CP v2 datasets. Experimental results show that our method achieves state-of-the-art on VQA-CP v2. Further analyses show that SQSs help build direct semantic connections between questions and images, provide question-adaptive variable-length reasoning chains, and with explicit interpretability as well as error traceability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑