arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2201.11794cs.CVcs.AIcs.CLcs.LG
收藏

基于知识图谱的视觉迁移学习研究综述

A Survey on Visual Transfer Learning using Knowledge Graphs

Robert Bosch GmbH(罗伯特·博世股份有限公司);Trier University(特里尔大学)

Sebastian Monka, Lavdim Halilaj, Achim Rettinger

摘要:

最近的计算机视觉方法利用深度学习方法,因为如果训练和测试域遵循相同的底层数据分布,则它们的性能相当好。然而,已经表明,在现实世界中使用这些方法时发生的图像中的微小变化可能导致不可预测的错误。迁移学习是机器学习试图防止这些错误的领域。特别是,近年来,使用编码在语言嵌入或知识图谱(KG)中的辅助知识来增强图像数据的方法已经取得了可喜的成果。本调查的重点是视觉迁移学习的方法,使用幼儿园。知识图可以表示辅助知识,或者在一个底层的图形结构的模式,或在一个基于向量的知识图谱嵌入。为了使读者能够在特定KG-DL配置的帮助下解决视觉迁移学习问题,我们首先描述了各种表达式的KG的相关建模结构,例如有向标记图,超图和超关系图。我们解释了特征提取器的概念,同时特别提到视觉和语义特征。我们提供了一个广泛的概述知识图谱嵌入方法,并描述了几个联合训练目标,适合将它们与高维视觉嵌入相结合。主要部分介绍了KG如何与DL管道相结合的四种不同类别:1)知识图谱作为评审员; 2)知识图谱作为受训者; 3)知识图谱作为培训师;以及4)知识图谱作为同行。为了帮助研究人员找到评估基准,我们提供了一个通用KG和一组图像处理数据集和基准,包括各种类型的辅助知识的概述。最后,我们总结了相关的调查,并提出了未来研究的挑战和开放问题的展望。

英文摘要:

Recent approaches of computer vision utilize deep learning methods as they perform quite well if training and testing domains follow the same underlying data distribution. However, it has been shown that minor variations in the images that occur when using these methods in the real world can lead to unpredictable errors. Transfer learning is the area of machine learning that tries to prevent these errors. Especially, approaches that augment image data using auxiliary knowledge encoded in language embeddings or knowledge graphs (KGs) have achieved promising results in recent years. This survey focuses on visual transfer learning approaches using KGs. KGs can represent auxiliary knowledge either in an underlying graph-structured schema or in a vector-based knowledge graph embedding. Intending to enable the reader to solve visual transfer learning problems with the help of specific KG-DL configurations we start with a description of relevant modeling structures of a KG of various expressions, such as directed labeled graphs, hypergraphs, and hyper-relational graphs. We explain the notion of feature extractor, while specifically referring to visual and semantic features. We provide a broad overview of knowledge graph embedding methods and describe several joint training objectives suitable to combine them with high dimensional visual embeddings. The main section introduces four different categories on how a KG can be combined with a DL pipeline: 1) Knowledge Graph as a Reviewer; 2) Knowledge Graph as a Trainee; 3) Knowledge Graph as a Trainer; and 4) Knowledge Graph as a Peer. To help researchers find evaluation benchmarks, we provide an overview of generic KGs and a set of image processing datasets and benchmarks including various types of auxiliary knowledge. Last, we summarize related surveys and give an outlook about challenges and open issues for future research.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑