arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2201.05575cs.CLcs.AIcs.DBcs.IRcs.LG

通过记忆进行推理:最近邻知识图谱嵌入

Reasoning Through Memorization: Nearest Neighbor Knowledge Graph Embeddings

Zhejiang University(浙江大学)

Peng Wang, Xin Xie, Xiaohan Wang, Ningyu Zhang

AI总结:

提出kNN-KGE方法,通过预训练语言模型结合k近邻线性插值实体分布,显式记忆罕见或新实体,显著提升归纳与直推式链接预测及低资源场景下的性能。

摘要:

先前的知识图谱嵌入方法通常将实体映射为表示并利用评分函数预测目标实体,但它们通常难以对罕见或新出现的未见实体进行推理。在本文中,我们提出了kNN-KGE,这是一种结合预训练语言模型的新知识图谱嵌入方法,通过将其实体分布与k近邻进行线性插值来实现。我们根据知识库中实体嵌入空间的距离来计算最近邻。我们的方法允许罕见或新出现的实体被显式记忆,而不是隐式存储在模型参数中。实验结果表明,我们的方法可以改进归纳和直推式链接预测结果,并在仅有少量三元组的低资源设置下产生更好的性能,这可能更容易通过显式记忆进行推理。代码可在https://github.com/zjunlp/KNN-KG获取。

英文摘要:

Previous knowledge graph embedding approaches usually map entities to representations and utilize score functions to predict the target entities, yet they typically struggle to reason rare or emerging unseen entities. In this paper, we propose kNN-KGE, a new knowledge graph embedding approach with pre-trained language models, by linearly interpolating its entity distribution with k-nearest neighbors. We compute the nearest neighbors based on the distance in the entity embedding space from the knowledge store. Our approach can allow rare or emerging entities to be memorized explicitly rather than implicitly in model parameters. Experimental results demonstrate that our approach can improve inductive and transductive link prediction results and yield better performance for low-resource settings with only a few triples, which might be easier to reason via explicit memory. Code is available at https://github.com/zjunlp/KNN-KG.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑