用正确的图像区域回答问题：一种视觉注意正则化方法
Answer Questions with Right Image Regions: A Visual Attention Regularization Approach
摘要：
视觉问答（VQA）中的视觉注意定位正确的图像区域的答案预测，提供了一个强大的技术，以促进多模态的理解。然而，近年来的研究指出，视觉注意中突出显示的图像区域往往与给定的问题和答案无关，从而导致正确的视觉推理的模型混淆。为了解决这个问题，现有的方法大多采取对齐视觉注意力权重与人类的注意力。然而，收集这些人类数据既费力又昂贵，使得在数据集之间调整开发良好的模型变得繁重。为了解决这个问题，在本文中，我们设计了一种新的视觉注意正则化方法，即AttReg，更好地在VQA视觉接地。具体地，AttReg首先识别对于问答而言必不可少但意外地被忽略的图像区域（即，分配有低注意力权重）。然后，利用掩模引导的学习方案来规则化视觉注意力，使其更多地关注这些被忽略的关键区域。该方法是非常灵活和模型无关的，它可以集成到大多数基于视觉注意力的VQA模型，不需要人的注意力监督。在三个基准数据集上进行了广泛的实验，即，已执行VQA-CP v2、VQA-CP v1和VQA v2，以评价AttReg的有效性。作为一个副产品，当将AttReg纳入强基线LMH时，我们的方法可以实现60.00%的最新准确度，在VQA-CP v2基准数据集上的绝对性能增益为7.01%。
英文摘要：
Visual attention in Visual Question Answering (VQA) targets at locating the right image regions regarding the answer prediction, offering a powerful technique to promote multi-modal understanding. However, recent studies have pointed out that the highlighted image regions from the visual attention are often irrelevant to the given question and answer, leading to model confusion for correct visual reasoning. To tackle this problem, existing methods mostly resort to aligning the visual attention weights with human attentions. Nevertheless, gathering such human data is laborious and expensive, making it burdensome to adapt well-developed models across datasets. To address this issue, in this paper, we devise a novel visual attention regularization approach, namely AttReg, for better visual grounding in VQA. Specifically, AttReg firstly identifies the image regions which are essential for question answering yet unexpectedly ignored (i.e., assigned with low attention weights) by the backbone model. And then a mask-guided learning scheme is leveraged to regularize the visual attention to focus more on these ignored key regions. The proposed method is very flexible and model-agnostic, which can be integrated into most visual attention-based VQA models and require no human attention supervision. Extensive experiments over three benchmark datasets, i.e., VQA-CP v2, VQA-CP v1, and VQA v2, have been conducted to evaluate the effectiveness of AttReg. As a by-product, when incorporating AttReg into the strong baseline LMH, our approach can achieve a new state-of-the-art accuracy of 60.00% with an absolute performance gain of 7.01% on the VQA-CP v2 benchmark dataset...
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
