arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1903.11207cs.CV
收藏

信息最大化可视化问题生成

Information Maximizing Visual Question Generation

Stanford University(斯坦福大学)

Ranjay Krishna, Michael Bernstein, Li Fei-Fei

摘要:

虽然图像到序列生成模型在人机通信中已经变得非常流行,但它们受到强烈支持安全通用问题的影响(“这张照片里是什么?").提出没有信息但相关的问题是不够的或有用的。我们认为,一个好的问题是一个有一个紧密集中的目的-一个旨在期待一种特定类型的反应。我们建立了一个模型,最大限度地提高图像,预期的答案和生成的问题之间的互信息。为了克服离散自然语言标记的不可微性,我们引入了一个变分的连续潜在空间上的预期答案的项目。我们用第二个潜在空间正则化这个潜在空间,以确保相似答案的聚类。即使我们不知道预期的答案,第二个潜在空间也可以生成目标驱动的问题,专门针对提取对象(“这个人扔的是什么”),属性(“这个人穿的是什么样的衬衫?“),color(“飞盘是什么颜色的?“),material(“飞盘是什么材料的?我们定量地表明,我们的模型能够保留有关预期答案类别的信息,从而产生更多样化的目标驱动问题。我们在一组真实世界的图像上启动我们的模型,并提取以前看不见的视觉概念。

英文摘要:

Though image-to-sequence generation models have become overwhelmingly popular in human-computer communications, they suffer from strongly favoring safe generic questions ("What is in this picture?"). Generating uninformative but relevant questions is not sufficient or useful. We argue that a good question is one that has a tightly focused purpose --- one that is aimed at expecting a specific type of response. We build a model that maximizes mutual information between the image, the expected answer and the generated question. To overcome the non-differentiability of discrete natural language tokens, we introduce a variational continuous latent space onto which the expected answers project. We regularize this latent space with a second latent space that ensures clustering of similar answers. Even when we don't know the expected answer, this second latent space can generate goal-driven questions specifically aimed at extracting objects ("what is the person throwing"), attributes, ("What kind of shirt is the person wearing?"), color ("what color is the frisbee?"), material ("What material is the frisbee?"), etc. We quantitatively show that our model is able to retain information about an expected answer category, resulting in more diverse, goal-driven questions. We launch our model on a set of real world images and extract previously unseen visual concepts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑