arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2110.04042cs.CV
收藏

Context-LGM:利用对象-上下文关系实现上下文感知的对象识别

Context-LGM: Leveraging Object-Context Relation for Context-Aware Object Recognition

Peking University(北京大学);Microsoft Research Asian(微软亚洲研究院);DeepWise AI Lab(深度智耀人工智能实验室)

Mingzhou Liu, Xinwei Sun, Fandong Zhang, Yizhou Yu, Yizhou Wang

摘要:

上下文是指与感兴趣对象相关的情境因素,它可以帮助在视觉识别中推断对象的状态或属性。由于这样的上下文特征太多样化(跨实例)被注释,现有的尝试简单地利用图像标签作为监督来学习它们,从而导致各种上下文技巧,例如特征金字塔,上下文注意力等。然而,如果没有仔细建模上下文的属性,特别是其与对象的关系,则其估计的上下文可能会受到很大的不准确性的影响。为了解决这个问题,我们提出了一种新的上下文潜在生成模型(上下文LGM),它考虑了对象上下文关系,并以分层的方式进行建模。具体来说,我们首先引入一个潜在的生成模型与一对相关的潜变量分别建模的对象和上下文,并嵌入其相关性通过生成过程。然后,为了推断上下文特征,我们重新制定了变分自动编码器(VAE)的目标函数,其中上下文特征被学习为以对象为条件的后验分布。最后,为了实现这种上下文后验,我们引入了一个Transformer,它以对象的信息为参考,定位相关的上下文因素。我们的方法的有效性验证了国家的最先进的性能上的两个上下文感知的对象识别任务,即肺癌预测和情感识别。

英文摘要:

Context, as referred to situational factors related to the object of interest, can help infer the object's states or properties in visual recognition. As such contextual features are too diverse (across instances) to be annotated, existing attempts simply exploit image labels as supervision to learn them, resulting in various contextual tricks, such as features pyramid, context attention, etc. However, without carefully modeling the context's properties, especially its relation to the object, their estimated context can suffer from large inaccuracy. To amend this problem, we propose a novel Contextual Latent Generative Model (Context-LGM), which considers the object-context relation and models it in a hierarchical manner. Specifically, we firstly introduce a latent generative model with a pair of correlated latent variables to respectively model the object and context, and embed their correlation via the generative process. Then, to infer contextual features, we reformulate the objective function of Variational Auto-Encoder (VAE), where contextual features are learned as a posterior distribution conditioned on the object. Finally, to implement this contextual posterior, we introduce a Transformer that takes the object's information as a reference and locates correlated contextual factors. The effectiveness of our method is verified by state-of-the-art performance on two context-aware object recognition tasks, i.e. lung cancer prediction and emotion recognition.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑