通过指定答案分布的熵来改写视觉问题
Rephrasing visual questions by specifying the entropy of the answer distribution
摘要:
视觉问答(VQA)是回答一个由问题和图像组成的视觉问题的任务。有些视觉问题是模糊的,有些是明确的,根据不同的情况改变问题的模糊性可能是合适的。然而,这个问题还没有得到解决的任何先前的工作。我们提出了一个新的任务,通过控制问题的歧义来改写问题。通过使用由VQA模型预测的答案分布的熵来定义视觉问题的模糊性。所提出的模型重新措辞的源问题与图像,使重新措辞的问题具有用户指定的歧义(或熵)。我们提出了两种学习策略来训练所提出的模型与VQA v2数据集,它没有歧义信息。我们证明了我们的方法的优势,可以控制的歧义的改写的问题,和一个有趣的观察,这是很难增加比减少歧义。
英文摘要:
Visual question answering (VQA) is a task of answering a visual question that is a pair of question and image. Some visual questions are ambiguous and some are clear, and it may be appropriate to change the ambiguity of questions from situation to situation. However, this issue has not been addressed by any prior work. We propose a novel task, rephrasing the questions by controlling the ambiguity of the questions. The ambiguity of a visual question is defined by the use of the entropy of the answer distribution predicted by a VQA model. The proposed model rephrases a source question given with an image so that the rephrased question has the ambiguity (or entropy) specified by users. We propose two learning strategies to train the proposed model with the VQA v2 dataset, which has no ambiguity information. We demonstrate the advantage of our approach that can control the ambiguity of the rephrased questions, and an interesting observation that it is harder to increase than to reduce ambiguity.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。