arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2201.11697cs.CVcs.AI
收藏

基于约束结构学习的场景图生成

Constrained Structure Learning for Scene Graph Generation

University of Surrey(萨里大学)

Daqi Liu, Miroslaw Bober, Josef Kittler

摘要:

作为一种结构化的预测任务,场景图生成的目的是建立一个可视化的场景图,以显式地建模输入图像中的对象及其关系。目前,均值场变分贝叶斯框架是现有方法所使用的事实上的方法,其中无约束推理步骤通常由消息传递神经网络来实现。然而,这种提法未能探索其他推理策略,并在很大程度上忽略了更一般的约束优化模型。在本文中,我们提出了一个约束结构学习方法,并提出了一个显式约束变分推理目标。而不是应用无处不在的消息传递策略,一个通用的约束优化方法-熵镜像下降-被用来解决约束变分推理步骤。我们在各种流行的场景图生成基准上验证了所提出的通用模型,并表明它优于最先进的方法。

英文摘要:

As a structured prediction task, scene graph generation aims to build a visually-grounded scene graph to explicitly model objects and their relationships in an input image. Currently, the mean field variational Bayesian framework is the de facto methodology used by the existing methods, in which the unconstrained inference step is often implemented by a message passing neural network. However, such formulation fails to explore other inference strategies, and largely ignores the more general constrained optimization models. In this paper, we present a constrained structure learning method, for which an explicit constrained variational inference objective is proposed. Instead of applying the ubiquitous message-passing strategy, a generic constrained optimization method - entropic mirror descent - is utilized to solve the constrained variational inference step. We validate the proposed generic model on various popular scene graph generation benchmarks and show that it outperforms the state-of-the-art methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑