arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2109.10376cs.NEcs.AIcs.LGq-bio.NC
收藏

通过结构学习:迈向深度神经形态知识图谱嵌入

Learning through structure: towards deep neuromorphic knowledge graph embeddings

Siemens AG Technology(西门子股份公司技术部)

Victor Caceres Chian, Marcel Hildebrandt, Thomas Runkler, Dominik Dold

AI总结:

本研究提出将知识图谱推理的深度图学习架构映射到神经形态架构的策略,结合冻结图神经网络与浅层嵌入模型,实现传统硬件上的提速降存,还扩展出适用于神经形态硬件的脉冲神经网络版本的稀疏嵌入算法。

摘要:

计算图结构数据的潜在表示是众多工业与学术应用中普遍存在的学习任务,涵盖分子合成、社交网络分析以及推荐系统等领域。知识图谱是与语义网相关的最受欢迎且应用最广泛的数据表示形式之一。除了以机器可读的格式构建事实知识外,知识图谱还是众多人工智能应用的核心支撑,能够将上下文信息融入各类学习算法中。图神经网络试图通过相邻节点间的消息传递启发式方法,将图结构编码到低维向量空间中。近年来,多种不同的图神经网络架构在众多学习任务中展现出了突破性的性能。在本研究中,我们提出了一种将用于知识图谱推理的深度图学习架构映射到神经形态架构的策略。基于随机初始化且未训练(即冻结)的图神经网络能够保留局部图结构这一洞见,我们将一个冻结的神经网络与浅层知识图谱嵌入模型相结合。我们通过实验证明,即便在传统计算硬件上,这种方法也能在保持具有竞争力的性能水平的同时,实现显著的速度提升与内存占用减少。此外,我们将该冻结架构扩展到脉冲神经网络,提出了一种新颖的、基于事件的高度稀疏知识图谱嵌入算法,该算法适用于在神经形态硬件中实现。

英文摘要:

Computing latent representations for graph-structured data is an ubiquitous learning task in many industrial and academic applications ranging from molecule synthetization to social network analysis and recommender systems. Knowledge graphs are among the most popular and widely used data representations related to the Semantic Web. Next to structuring factual knowledge in a machine-readable format, knowledge graphs serve as the backbone of many artificial intelligence applications and allow the ingestion of context information into various learning algorithms. Graph neural networks attempt to encode graph structures in low-dimensional vector spaces via a message passing heuristic between neighboring nodes. Over the recent years, a multitude of different graph neural network architectures demonstrated ground-breaking performances in many learning tasks. In this work, we propose a strategy to map deep graph learning architectures for knowledge graph reasoning to neuromorphic architectures. Based on the insight that randomly initialized and untrained (i.e., frozen) graph neural networks are able to preserve local graph structures, we compose a frozen neural network with shallow knowledge graph embedding models. We experimentally show that already on conventional computing hardware, this leads to a significant speedup and memory reduction while maintaining a competitive performance level. Moreover, we extend the frozen architecture to spiking neural networks, introducing a novel, event-based and highly sparse knowledge graph embedding algorithm that is suitable for implementation in neuromorphic hardware.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑