arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1809.06213cs.CVcs.CLcs.LGstat.ML
收藏

基于上下文相关扩散网络的视觉关系检测

Context-Dependent Diffusion Network for Visual Relationship Detection

Nanjing University of Science and Technology(南京理工大学);Southeast University(东南大学)

Zhen Cui, Chunyan Xu, Wenming Zheng, Jian Yang

摘要:

视觉关系检测是计算机视觉和自然语言之间的桥梁,可用于图像的场景理解。与单纯的宾语识别任务不同,主谓宾关系三元组位于极端多样性空间,如\textit{person-behind-person}和\textit{car-behind-building},存在组合爆炸问题。在本文中,我们提出了一个上下文相关的扩散网络(CDDN)框架来处理视觉关系检测。为了捕捉不同对象实例之间的交互,构建了两种类型的图,词语义图和视觉场景图,以编码全局上下文相关性。语义图是通过语言先验建立的,以模拟对象之间的语义相关性,而视觉场景图定义了场景对象的连接,以便利用周围的场景信息。针对图结构数据,设计了一种扩散网络自适应地聚合上下文信息,利用其与图的同构不变性,有效地学习视觉关系的潜在表征,满足视觉关系检测的需要.在两个广泛使用的数据集上的实验表明,我们提出的方法是更有效的,达到了最先进的性能。

英文摘要:

Visual relationship detection can bridge the gap between computer vision and natural language for scene understanding of images. Different from pure object recognition tasks, the relation triplets of subject-predicate-object lie on an extreme diversity space, such as \textit{person-behind-person} and \textit{car-behind-building}, while suffering from the problem of combinatorial explosion. In this paper, we propose a context-dependent diffusion network (CDDN) framework to deal with visual relationship detection. To capture the interactions of different object instances, two types of graphs, word semantic graph and visual scene graph, are constructed to encode global context interdependency. The semantic graph is built through language priors to model semantic correlations across objects, whilst the visual scene graph defines the connections of scene objects so as to utilize the surrounding scene information. For the graph-structured data, we design a diffusion network to adaptively aggregate information from contexts, which can effectively learn latent representations of visual relationships and well cater to visual relationship detection in view of its isomorphic invariance to graphs. Experiments on two widely-used datasets demonstrate that our proposed method is more effective and achieves the state-of-the-art performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑