利用细粒度因果干预消除视觉问答中的语言偏差
Eliminating the Language Bias for Visual Question Answering with fine-grained Causal Intervention
AI总结：
针对VQA中现有方法忽视细粒度信息导致无法充分捕捉语言偏差的问题，提出CIBi因果干预方案，将偏差分为上下文和关键词两类，结合对比学习与反事实生成分别消除，提升了多模态表征与模型性能。
摘要：
尽管视觉问答（VQA）取得了显著进展，但缓解由文本信息引入的语言偏差这一挑战仍未解决。以往的方法从粗粒度角度捕捉语言偏差。然而，句子内部更细粒度的信息（如上下文和关键词）会导致不同的偏差。由于忽视了细粒度信息，大多数现有方法无法充分捕捉语言偏差。在本文中，我们提出了一种名为CIBi的新型因果干预训练方案，从更细粒度的视角消除语言偏差。具体而言，我们将语言偏差分为上下文偏差和关键词偏差。我们采用因果干预和对比学习来消除上下文偏差并改善多模态表征。此外，我们基于反事实生成设计了一个新的仅问题分支，以蒸馏并消除关键词偏差。实验结果表明，CIBi适用于各种VQA模型，取得了有竞争力的性能。
英文摘要：
Despite the remarkable advancements in Visual Question Answering (VQA), the challenge of mitigating the language bias introduced by textual information remains unresolved. Previous approaches capture language bias from a coarse-grained perspective. However, the finer-grained information within a sentence, such as context and keywords, can result in different biases. Due to the ignorance of fine-grained information, most existing methods fail to sufficiently capture language bias. In this paper, we propose a novel causal intervention training scheme named CIBi to eliminate language bias from a finer-grained perspective. Specifically, we divide the language bias into context bias and keyword bias. We employ causal intervention and contrastive learning to eliminate context bias and improve the multi-modal representation. Additionally, we design a new question-only branch based on counterfactual generation to distill and eliminate keyword bias. Experimental results illustrate that CIBi is applicable to various VQA models, yielding competitive performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
