arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2106.12144cs.CLcs.AIcs.LG
收藏

NodePiece:大型知识图谱的组合和参数有效表示

NodePiece: Compositional and Parameter-Efficient Representations of Large Knowledge Graphs

Mila(米拉);McGill University(麦吉尔大学)

Mikhail Galkin, Etienne Denis, Jiapeng Wu, William L. Hamilton

摘要:

传统的知识图谱(KG)的表示学习算法将每个实体映射到唯一的嵌入向量。这样的浅查找导致用于存储嵌入矩阵的存储器消耗的线性增长,并且当与真实世界KG一起工作时引起高计算成本。与NLP中常用的子词标记化相似,我们探索了可能具有次线性内存要求的更参数有效的节点嵌入策略。为此,我们提出了NodePiece,一种基于锚的方法来学习固定大小的实体词汇。在NodePiece中,子词/子实体单元的词汇表是从具有已知关系类型的图中的锚节点构建的。给定这样一个固定大小的词汇表,可以为任何实体引导编码和嵌入,包括那些在训练过程中看不到的实体。实验表明,NodePiece在节点分类,链接预测和关系预测任务中表现出竞争力,同时保留图中不到10%的显式节点作为锚点,并且通常具有10倍的参数。为此,我们证明了一个启用NodePiece的模型在一个参数少70倍的大型OGB WikiKG 2图上的性能优于现有的浅层模型。

英文摘要:

Conventional representation learning algorithms for knowledge graphs (KG) map each entity to a unique embedding vector. Such a shallow lookup results in a linear growth of memory consumption for storing the embedding matrix and incurs high computational costs when working with real-world KGs. Drawing parallels with subword tokenization commonly used in NLP, we explore the landscape of more parameter-efficient node embedding strategies with possibly sublinear memory requirements. To this end, we propose NodePiece, an anchor-based approach to learn a fixed-size entity vocabulary. In NodePiece, a vocabulary of subword/sub-entity units is constructed from anchor nodes in a graph with known relation types. Given such a fixed-size vocabulary, it is possible to bootstrap an encoding and embedding for any entity, including those unseen during training. Experiments show that NodePiece performs competitively in node classification, link prediction, and relation prediction tasks while retaining less than 10% of explicit nodes in a graph as anchors and often having 10x fewer parameters. To this end, we show that a NodePiece-enabled model outperforms existing shallow models on a large OGB WikiKG 2 graph having 70x fewer parameters.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑