arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1904.00560cs.CV
收藏

基于外部知识的场景图生成与图像重建

Scene Graph Generation with External Knowledge and Image Reconstruction

Nanyang Technological University(南洋理工大学);Adobe Research(奥多比研究院);University of Georgia(佐治亚大学);Google Cloud AI(谷歌云人工智能)

Jiuxiang Gu, Handong Zhao, Zhe Lin, Sheng Li, Jianfei Cai, Mingyang Ling

摘要:

随着目标检测、属性和关系预测、~ \etc等图像理解任务的发展,场景图生成得到了越来越多的关注。然而,现有的数据集在对象和关系标签方面存在偏差,或者经常带有噪声和缺失的注释,这使得开发可靠的场景图预测模型非常具有挑战性。在本文中,我们提出了一种新的场景图生成算法与外部知识和图像重建损失,以克服这些数据集的问题。特别是,我们从外部知识库中提取常识知识,以改进场景图生成中的对象和短语特征,以提高泛化能力。为了解决噪声对象注释的偏差,我们引入了辅助图像重建路径来正则化场景图生成网络。大量的实验表明,我们的框架可以生成更好的场景图,在两个基准数据集上实现了最先进的性能:视觉关系检测和视觉基因组数据集。

英文摘要:

Scene graph generation has received growing attention with the advancements in image understanding tasks such as object detection, attributes and relationship prediction,~\etc. However, existing datasets are biased in terms of object and relationship labels, or often come with noisy and missing annotations, which makes the development of a reliable scene graph prediction model very challenging. In this paper, we propose a novel scene graph generation algorithm with external knowledge and image reconstruction loss to overcome these dataset issues. In particular, we extract commonsense knowledge from the external knowledge base to refine object and phrase features for improving generalizability in scene graph generation. To address the bias of noisy object annotations, we introduce an auxiliary image reconstruction path to regularize the scene graph generation network. Extensive experiments show that our framework can generate better scene graphs, achieving the state-of-the-art performance on two benchmark datasets: Visual Relationship Detection and Visual Genome datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑