arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1708.04320cs.CV
收藏

基于图神经网络的态势识别

Situation Recognition with Graph Neural Networks

The Chinese University of Hong Kong(香港中文大学);University of Toronto(多伦多大学);Uber Advanced Technologies Group(优步先进技术集团);Youtu Lab, Tencent(腾讯优图实验室);Vector Institute(向量学院)

Ruiyu Li, Makarand Tapaswi, Renjie Liao, Jiaya Jia, Raquel Urtasun, Sanja Fidler

摘要:

我们解决了识别图像中的情况的问题。给定一幅图像,任务是预测最突出的动词(动作),并填充其语义角色,如谁在执行动作,动作的来源和目标是什么等。不同的动词有不同的角色(例如攻击有武器),每个角色可以采取许多可能的值(名词)。我们提出了一个基于图神经网络的模型,使我们能够使用图上定义的神经网络有效地捕获角色之间的联合依赖关系。不同的图形连接的实验表明,我们的方法,传播角色之间的信息显着优于现有的工作,以及多个基线。在预测整个情况方面,我们比以前的工作提高了大约3-5%。我们还提供了一个全面的定性分析我们的模型和不同的角色在动词的影响。

英文摘要:

We address the problem of recognizing situations in images. Given an image, the task is to predict the most salient verb (action), and fill its semantic roles such as who is performing the action, what is the source and target of the action, etc. Different verbs have different roles (e.g. attacking has weapon), and each role can take on many possible values (nouns). We propose a model based on Graph Neural Networks that allows us to efficiently capture joint dependencies between roles using neural networks defined on a graph. Experiments with different graph connectivities show that our approach that propagates information between roles significantly outperforms existing work, as well as multiple baselines. We obtain roughly 3-5% improvement over previous work in predicting the full situation. We also provide a thorough qualitative analysis of our model and influence of different roles in the verbs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑