arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.00107cs.CVcs.CLcs.LG
收藏

图像集可视化问答分析

Analysis on Image Set Visual Question Answering

Abhinav Khattar, Aviral Joshi, Har Simrat Singh, Pulkit Goel, Rohit Prakash Barnwal

摘要:

我们解决了视觉问答的挑战,在多图像设置的ISVQA数据集。传统的VQA任务集中在单图像设置上,其中从单个图像生成目标答案。然而,图像集VQA由一组图像组成,需要找到图像之间的连接,基于这些连接将图像中的对象关联起来,并生成统一的答案。在本报告中,我们使用4种方法来提高任务的性能。我们分析和比较我们的结果与三个基线模型- LXMERT,HME-VideoQA和VisualBERT -并表明我们的方法可以提供一个轻微的改善基线。具体来说,我们尝试提高模型的空间感知能力,并使用增强的预训练帮助模型识别颜色,使用对抗正则化减少语言依赖,并使用回归损失和基于图的重复数据删除来改进计数。我们进一步深入分析了ISVQA数据集中的语言偏见,并展示了在ISVQA上训练的模型如何隐式地学习将语言与最终答案更紧密地联系起来。

英文摘要:

We tackle the challenge of Visual Question Answering in multi-image setting for the ISVQA dataset. Traditional VQA tasks have focused on a single-image setting where the target answer is generated from a single image. Image set VQA, however, comprises of a set of images and requires finding connection between images, relate the objects across images based on these connections and generate a unified answer. In this report, we work with 4 approaches in a bid to improve the performance on the task. We analyse and compare our results with three baseline models - LXMERT, HME-VideoQA and VisualBERT - and show that our approaches can provide a slight improvement over the baselines. In specific, we try to improve on the spatial awareness of the model and help the model identify color using enhanced pre-training, reduce language dependence using adversarial regularization, and improve counting using regression loss and graph based deduplication. We further delve into an in-depth analysis on the language bias in the ISVQA dataset and show how models trained on ISVQA implicitly learn to associate language more strongly with the final answer.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑