我们是否在MovieQA中提出了正确的问题?
Are we asking the right questions in MovieQA?
摘要:
视觉和语言的联合任务,如视觉问答,很吸引人,因为它们探索了高层次的理解,但与此同时,更容易出现语言偏见。在本文中,我们探讨了MovieQA数据集中的偏差,并提出了一个可以利用它们的非常简单的模型。我们发现,使用正确的词嵌入是至关重要的。通过使用经过适当训练的词嵌入,大约一半的问答(QA)可以通过单独查看问题和答案来回答,完全忽略视频剪辑,字幕和电影剧本的叙事背景。与排行榜上最好的已发表论文相比,我们的简单问题+答案模型将视频+字幕类别的准确率提高了5%,字幕提高了5%,DVS提高了15%,脚本提高了6%。
英文摘要:
Joint vision and language tasks like visual question answering are fascinating because they explore high-level understanding, but at the same time, can be more prone to language biases. In this paper, we explore the biases in the MovieQA dataset and propose a strikingly simple model which can exploit them. We find that using the right word embedding is of utmost importance. By using an appropriately trained word embedding, about half the Question-Answers (QAs) can be answered by looking at the questions and answers alone, completely ignoring narrative context from video clips, subtitles, and movie scripts. Compared to the best published papers on the leaderboard, our simple question + answer only model improves accuracy by 5% for video + subtitle category, 5% for subtitle, 15% for DVS and 6% higher for scripts.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。