揭示视觉问答中的跨模态偏差：一种基于可能世界VQA的因果视角
Unveiling Cross Modality Bias in Visual Question Answering: A Causal View with Possible Worlds VQA
AI总结：
本文提出一种基于反事实推理的因果方法，同时建模并消除VQA中视觉和语言的混杂偏差，并在VQA-CP v2上取得领先性能。
摘要：
为了提高VQA系统的泛化能力，许多近期研究试图消除那些将问题或图像捷径到答案的虚假语言或视觉关联偏差。尽管做出了这些努力，现有文献未能同时解决视觉和语言的混杂效应。因此，当它们减少从一种模态学到的偏差时，通常会增加来自另一种模态的偏差。在本文中，我们首先对同时导致语言和视觉偏差的混杂效应进行建模，然后提出一种反事实推理来消除该效应的影响。以这种策略训练的模型能够同时且高效地减少视觉和语言偏差。据我们所知，这是第一项利用因果解释消除关系来减少VQA中视觉和语言混杂效应所导致偏差的工作。我们为该方法配套了一种解释消除策略，与现有方法相比，推动了数值答案问题准确率的提升，而这一问题一直是一个开放难题。所提出的方法在VQA-CP v2数据集上优于最先进的方法。
英文摘要：
To increase the generalization capability of VQA systems, many recent studies have tried to de-bias spurious language or vision associations that shortcut the question or image to the answer. Despite these efforts, the literature fails to address the confounding effect of vision and language simultaneously. As a result, when they reduce bias learned from one modality, they usually increase bias from the other. In this paper, we first model a confounding effect that causes language and vision bias simultaneously, then propose a counterfactual inference to remove the influence of this effect. The model trained in this strategy can concurrently and efficiently reduce vision and language bias. To the best of our knowledge, this is the first work to reduce biases resulting from confounding effects of vision and language in VQA, leveraging causal explain-away relations. We accompany our method with an explain-away strategy, pushing the accuracy of the questions with numerical answers results compared to existing methods that have been an open problem. The proposed method outperforms the state-of-the-art methods in VQA-CP v2 datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
