arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2112.04087cs.AI
收藏

基于结构上下文预训练的知识图谱表示学习改进

Improving Knowledge Graph Representation Learning by Structure Contextual Pre-training

Zhejiang University(浙江大学);Alibaba Group(阿里巴巴集团);Zhejiang University & AZFT JointLab for Knowledge Engine(浙江大学与AZFT知识引擎联合实验室)

Ganqiang Ye, Wen Zhang, Zhen Bi, Chi Man Wong, Chen Hui, Huajun Chen

摘要:

知识图谱(KG)的表示学习模型已被证明在编码结构信息和执行KG上的推理方面是有效的。在本文中,我们提出了一种新的预训练,然后微调知识图谱表示学习,其中KG模型首先预训练与三重分类任务,然后区分微调特定的下游任务,如实体类型预测和实体对齐。借鉴在典型的预训练语言模型中学习深度上下文化单词表示的一般思想,我们提出SCoP来学习具有目标三元组编码的结构和上下文三元组的预训练KG表示。实验结果表明,微调SCoP不仅优于基线的下游任务的组合的结果,但也避免了繁琐的特定于任务的模型设计和参数训练。

英文摘要:

Representation learning models for Knowledge Graphs (KG) have proven to be effective in encoding structural information and performing reasoning over KGs. In this paper, we propose a novel pre-training-then-fine-tuning framework for knowledge graph representation learning, in which a KG model is firstly pre-trained with triple classification task, followed by discriminative fine-tuning on specific downstream tasks such as entity type prediction and entity alignment. Drawing on the general ideas of learning deep contextualized word representations in typical pre-trained language models, we propose SCoP to learn pre-trained KG representations with structural and contextual triples of the target triple encoded. Experimental results demonstrate that fine-tuning SCoP not only outperforms results of baselines on a portfolio of downstream tasks but also avoids tedious task-specific model design and parameter training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑