arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1708.08310cs.LGcs.CVstat.ML
收藏

基于知识图谱的开放世界视觉识别

Open-World Visual Recognition Using Knowledge Graphs

IBM Research – Ireland(IBM研究院爱尔兰)

Vincent P. A. Lonij, Ambrish Rawat, Maria-Irina Nicolae

摘要:

在现实世界中,视觉识别系统可以对属于先前未知类别标签的图像进行预测。为了使语义有意义的预测,这样的输入,我们提出了一个两步的方法,利用信息从知识图谱。首先,学习知识图表示以将大量实体嵌入语义空间。其次,学习图像表示以将图像嵌入到同一空间中。在这种设置下,我们能够预测任何开放世界图像的关系三元组形式的结构化属性。即使在从知识图谱和图像嵌入的训练协议中省略了一组标签时也是如此。此外,我们附加适当的平滑约束的学习框架,并显示如何先验知识可以纳入模型。与我们的基线相比,这两项改进将视觉识别的性能提高了六倍。最后,我们提出了一个新的,扩展的数据集,我们使用的实验。

英文摘要:

In a real-world setting, visual recognition systems can be brought to make predictions for images belonging to previously unknown class labels. In order to make semantically meaningful predictions for such inputs, we propose a two-step approach that utilizes information from knowledge graphs. First, a knowledge-graph representation is learned to embed a large set of entities into a semantic space. Second, an image representation is learned to embed images into the same space. Under this setup, we are able to predict structured properties in the form of relationship triples for any open-world image. This is true even when a set of labels has been omitted from the training protocols of both the knowledge graph and image embeddings. Furthermore, we append this learning framework with appropriate smoothness constraints and show how prior knowledge can be incorporated into the model. Both these improvements combined increase performance for visual recognition by a factor of six compared to our baseline. Finally, we propose a new, extended dataset which we use for experiments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑