arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2210.11233cs.AIcs.CLcs.CVcs.LGcs.SC
收藏

基于知识图谱的上下文驱动视觉目标识别

Context-driven Visual Object Recognition based on Knowledge Graphs

Bosch Center for Artificial Intelligence(博世人工智能中心);Trier University(特里尔大学)

Sebastian Monka, Lavdim Halilaj, Achim Rettinger

摘要:

目前用于对象识别的深度学习方法是纯数据驱动的,需要大量的训练样本才能达到良好的效果。由于它们仅依赖于图像数据,这些方法在面对即使发生小偏差的新环境时往往会失败。然而,人类感知已被证明对这种分布偏移显著更鲁棒。据推测,他们的能力,以处理未知的情况下,是基于广泛纳入的上下文知识。上下文可以基于场景中的对象共现,也可以基于经验记忆。根据人类视觉皮层使用上下文来为所看到的图像形成不同的对象表示,我们提出了一种方法,通过使用编码在知识图谱中的外部上下文知识来增强深度学习方法。因此,我们从一个通用的知识图谱中提取不同的上下文视图,将视图转换到向量空间中,并将其注入DNN。我们进行了一系列的实验,以研究不同的上下文视图对同一图像数据集的学习对象表示的影响。实验结果提供了证据表明,上下文视图影响DNN中的图像表示不同,因此导致对相同图像的不同预测。我们还表明,上下文有助于加强对象识别模型的鲁棒性的分布图像,通常发生在迁移学习任务或现实世界的情况。

英文摘要:

Current deep learning methods for object recognition are purely data-driven and require a large number of training samples to achieve good results. Due to their sole dependence on image data, these methods tend to fail when confronted with new environments where even small deviations occur. Human perception, however, has proven to be significantly more robust to such distribution shifts. It is assumed that their ability to deal with unknown scenarios is based on extensive incorporation of contextual knowledge. Context can be based either on object co-occurrences in a scene or on memory of experience. In accordance with the human visual cortex which uses context to form different object representations for a seen image, we propose an approach that enhances deep learning methods by using external contextual knowledge encoded in a knowledge graph. Therefore, we extract different contextual views from a generic knowledge graph, transform the views into vector space and infuse it into a DNN. We conduct a series of experiments to investigate the impact of different contextual views on the learned object representations for the same image dataset. The experimental results provide evidence that the contextual views influence the image representations in the DNN differently and therefore lead to different predictions for the same images. We also show that context helps to strengthen the robustness of object recognition models for out-of-distribution images, usually occurring in transfer learning tasks or real-world scenarios.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑