arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2006.05726cs.CVcs.CL
收藏

估计VQA回答空间的语义结构

Estimating semantic structure for the VQA answer space

Orange Labs(Orange实验室);Université de Lyon(里昂大学);INSA-Lyon(里昂国立应用科学学院);LIRIS(里昂信息与图像研究实验室);UMR CNRS 5205(法国国家科研中心联合研究单位5205)

Corentin Kervadec, Grigory Antipov, Moez Baccouche, Christian Wolf

摘要:

自视觉问答(VQA,即回答在图像上提出的问题)出现以来,一直被视为一组预定义答案的分类问题。尽管它很方便,但这种分类方法很难反映问题的语义,将回答限制在独立建议之间的选择,而不考虑它们之间的相似性(例如,同样惩罚回答猫或德国牧羊犬而不是狗)。我们解决这个问题,提出(1)两个措施之间的接近VQA类,和(2)相应的损失,考虑到估计的接近。这通过减少语言偏差显著提高了VQA模型的泛化能力。特别是,我们表明,我们的方法是完全模型不可知的,因为它允许与三个不同的VQA模型一致的改进。最后,通过将我们的方法与语言偏差减少方法相结合,我们在具有挑战性的VQAv 2-CP数据集上报告了SOTA级别的性能。

英文摘要:

Since its appearance, Visual Question Answering (VQA, i.e. answering a question posed over an image), has always been treated as a classification problem over a set of predefined answers. Despite its convenience, this classification approach poorly reflects the semantics of the problem limiting the answering to a choice between independent proposals, without taking into account the similarity between them (e.g. equally penalizing for answering cat or German shepherd instead of dog). We address this issue by proposing (1) two measures of proximity between VQA classes, and (2) a corresponding loss which takes into account the estimated proximity. This significantly improves the generalization of VQA models by reducing their language bias. In particular, we show that our approach is completely model-agnostic since it allows consistent improvements with three different VQA models. Finally, by combining our method with a language bias reduction approach, we report SOTA-level performance on the challenging VQAv2-CP dataset.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑