arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2012.05473cs.CV
收藏

基于张量合成网络的视觉关系预测

Tensor Composition Net for Visual Relationship Prediction

Nanjing University(南京大学);The University of Edinburgh(爱丁堡大学);Samsung AI Research Centre(三星人工智能研究中心)

Yuting Qiang, Yongxin Yang, Xueting Zhang, Yanwen Guo, Timothy M. Hospedales

摘要:

我们提出了一种新的张量组合网络(TCN)来预测图像中的视觉关系。视觉关系预测(VRP)提供了一个更具挑战性的测试图像理解比传统的图像标记,是很难学习,由于一个大的标签空间和不完整的注释。我们的TCN的关键思想是利用视觉关系张量的低秩属性,以便利用对象和关系内部和之间的相关性,并对图像中的所有视觉关系进行结构化预测。为了展示我们的模型的有效性,我们首先将我们的模型与多标签图像分类(MLIC)方法、极限多标签分类(XMC)方法和VRD方法进行了实证比较。然后,我们表明,由于我们的张量(去)组成层,我们的模型可以预测在训练数据集中没有看到的视觉关系。最后,我们展示了我们的TCN的图像级视觉关系预测提供了一个简单而有效的机制,基于关系的图像检索,即使与VRD方法相比。

英文摘要:

We present a novel Tensor Composition Net (TCN) to predict visual relationships in images. Visual Relationship Prediction (VRP) provides a more challenging test of image understanding than conventional image tagging and is difficult to learn due to a large label-space and incomplete annotation. The key idea of our TCN is to exploit the low-rank property of the visual relationship tensor, so as to leverage correlations within and across objects and relations and make a structured prediction of all visual relationships in an image. To show the effectiveness of our model, we first empirically compare our model with Multi-Label Image Classification (MLIC) methods, eXtreme Multi-label Classification (XMC) methods, and VRD methods. We then show that thanks to our tensor (de)composition layer, our model can predict visual relationships which have not been seen in the training dataset. We finally show our TCN's image-level visual relationship prediction provides a simple and efficient mechanism for relation-based image-retrieval even compared with VRD methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑