arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2211.02744cs.CLcs.AI
收藏

KGLM:在语言模型中集成知识图谱结构以进行链接预测

KGLM: Integrating Knowledge Graph Structure in Language Models for Link Prediction

University of California, Davis(加州大学戴维斯分校)

Jason Youn, Ilias Tagkopoulos

AI总结:

提出KGLM架构,通过引入新的实体/关系嵌入层学习知识图谱结构,并使用图谱三元组进一步预训练语言模型,在链接预测任务上取得最先进性能。

摘要:

知识图谱表示复杂关系的能力使其被广泛用于知识表示、问答和推荐系统等需求。知识图谱表示的信息通常不完整,因此需要知识图谱补全任务。预训练和微调的语言模型在这些任务中展现出潜力,尽管这些模型忽略了知识图谱中编码的内在信息,即实体和关系类型。在这项工作中,我们提出了知识图谱语言模型(KGLM)架构,其中引入了新的实体/关系嵌入层,该层学习区分独特的实体和关系类型,从而使模型能够学习知识图谱的结构。我们展示了使用从知识图谱提取的三元组,通过该附加嵌入层对语言模型进行进一步预训练,随后进行标准微调,在基准数据集上为链接预测任务设定了新的最先进性能。

英文摘要:

The ability of knowledge graphs to represent complex relationships at scale has led to their adoption for various needs including knowledge representation, question-answering, and recommendation systems. Knowledge graphs are often incomplete in the information they represent, necessitating the need for knowledge graph completion tasks. Pre-trained and fine-tuned language models have shown promise in these tasks although these models ignore the intrinsic information encoded in the knowledge graph, namely the entity and relation types. In this work, we propose the Knowledge Graph Language Model (KGLM) architecture, where we introduce a new entity/relation embedding layer that learns to differentiate distinctive entity and relation types, therefore allowing the model to learn the structure of the knowledge graph. In this work, we show that further pre-training the language models with this additional embedding layer using the triples extracted from the knowledge graph, followed by the standard fine-tuning phase sets a new state-of-the-art performance for the link prediction task on the benchmark datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑