arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1905.01595cs.CVcs.LG
收藏

探索视觉关系检测中的不确定关系

On Exploring Undetermined Relationships for Visual Relationship Detection

Key Laboratory of Complex Systems Modeling and Simulation(复杂系统建模与仿真重点实验室);Hangzhou Dianzi University(杭州电子科技大学);University of Sydney(悉尼大学)

Yibing Zhan, Jun Yu, Ting Yu, Dacheng Tao

摘要:

在视觉关系检测中,人类标记的关系可以被视为确定关系。然而,仍然存在大量未标记的数据,例如具有不太重要的关系甚至没有关系的对象对。我们将这些未标记但可能有用的数据称为未确定的关系。虽然大量的文献存在,很少有方法利用这些不确定的关系的视觉关系检测。在本文中,我们探讨了视觉关系检测的未确定的关系的有益影响。我们提出了一种新的基于多模态特征的不确定关系学习网络(MF-URLN),并在关系检测方面取得了很大的进步。详细地说,我们的MF-URLN自动生成不确定的关系,通过比较对象对与人类标注的数据,根据设计的标准。然后,MF-URLN提取和融合三个互补的情态动词:视觉,空间和语言情态动词的对象对的特征。此外,MF-URLN提出了两个相关的子网络:一个子网络决定确定的信心,和其他预测的关系。我们在两个数据集上评估MF-URLN:视觉关系检测(VRD)和视觉基因组(VG)数据集。与现有方法相比,实验结果验证了未确定关系的显著改进,例如,在VRD数据集上,前50个关系检测召回率从19.5%提高到23.9%。

英文摘要:

In visual relationship detection, human-notated relationships can be regarded as determinate relationships. However, there are still large amount of unlabeled data, such as object pairs with less significant relationships or even with no relationships. We refer to these unlabeled but potentially useful data as undetermined relationships. Although a vast body of literature exists, few methods exploit these undetermined relationships for visual relationship detection. In this paper, we explore the beneficial effect of undetermined relationships on visual relationship detection. We propose a novel multi-modal feature based undetermined relationship learning network (MF-URLN) and achieve great improvements in relationship detection. In detail, our MF-URLN automatically generates undetermined relationships by comparing object pairs with human-notated data according to a designed criterion. Then, the MF-URLN extracts and fuses features of object pairs from three complementary modals: visual, spatial, and linguistic modals. Further, the MF-URLN proposes two correlated subnetworks: one subnetwork decides the determinate confidence, and the other predicts the relationships. We evaluate the MF-URLN on two datasets: the Visual Relationship Detection (VRD) and the Visual Genome (VG) datasets. The experimental results compared with state-of-the-art methods verify the significant improvements made by the undetermined relationships, e.g., the top-50 relation detection recall improves from 19.5% to 23.9% on the VRD dataset.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑