arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2312.01837cs.CL
收藏

基于预训练语言模型的知识图谱补全中提示解耦嵌入方法

Prompting Disentangled Embeddings for Knowledge Graph Completion with Pre-trained Language Model

Powerchina Huadong Engineering Corporation Limited(中国电力建设集团华东勘测设计研究院有限公司);School of Computer Science, Hangzhou Dianzi University(杭州电子科技大学计算机学院);The University of Manchester(曼彻斯特大学);Zhejiang University(浙江大学);The University of Edinburgh(爱丁堡大学)

Yuxia Geng, Jiaoyan Chen, Yuhang Zeng, Zhuo Chen, Wen Zhang, Jeff Z. Pan, Yuxiang Wang, Xiaoliang Xu

AI总结:

本文提出PDKGC方法,通过在冻结的预训练语言模型上使用硬任务提示和解耦结构提示,将文本信息与图结构知识有效结合,在三个KGC数据集上取得了优于现有方法的性能。

摘要:

图结构和文本信息在知识图谱补全(KGC)中都起着关键作用。随着BERT等预训练语言模型(PLMs)的成功,它们已被应用于KGC的文本编码。然而,当前的方法大多倾向于微调PLMs,导致巨大的训练成本,并且对更大规模PLMs的可扩展性有限。相比之下,我们提出利用提示(prompts),在冻结的PLM上执行KGC,仅训练提示部分。据此,我们提出了一种新的KGC方法,名为PDKGC,包含两种提示——一种是硬任务提示(hard task prompt),用于使KGC任务适应PLM预训练的token预测任务;另一种是解耦结构提示(disentangled structure prompt),学习解耦的图表示,从而使PLM能够将更相关的结构知识与文本信息相结合。借助这两种提示,PDKGC分别构建了一个文本预测器和一个结构预测器,二者的结合实现了更全面的实体预测。在三个广泛使用的KGC数据集上的可靠评估表明,PDKGC通常优于包括最先进方法在内的基线模型,并且其各个组件都是有效的。我们的代码和数据可在 https://github.com/genggengcss/PDKGC 获取。

英文摘要:

Both graph structures and textual information play a critical role in Knowledge Graph Completion (KGC). With the success of Pre-trained Language Models (PLMs) such as BERT, they have been applied for text encoding for KGC. However, the current methods mostly prefer to fine-tune PLMs, leading to huge training costs and limited scalability to larger PLMs. In contrast, we propose to utilize prompts and perform KGC on a frozen PLM with only the prompts trained. Accordingly, we propose a new KGC method named PDKGC with two prompts -- a hard task prompt which is to adapt the KGC task to the PLM pre-training task of token prediction, and a disentangled structure prompt which learns disentangled graph representation so as to enable the PLM to combine more relevant structure knowledge with the text information. With the two prompts, PDKGC builds a textual predictor and a structural predictor, respectively, and their combination leads to more comprehensive entity prediction. Solid evaluation on three widely used KGC datasets has shown that PDKGC often outperforms the baselines including the state-of-the-art, and its components are all effective. Our codes and data are available at https://github.com/genggengcss/PDKGC.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑