arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.13085cs.CLcs.AI
收藏

多领域知识图谱协同预训练与提示调优以服务多样下游任务

Multi-domain Knowledge Graph Collaborative Pre-training and Prompt Tuning for Diverse Downstream Tasks

Zhejiang University(浙江大学);Ant Group(蚂蚁集团);Alibaba-Zhejiang University Joint Institute of Frontier Technology(阿里巴巴-浙江大学前沿技术联合研究院)

Yichi Zhang, Binbin Hu, Zhuo Chen, Lingbing Guo, Ziqi Liu, Zhiqiang Zhang, Lei Liang, Huajun Chen, Wen Zhang

AI总结:

针对现有知识图谱预训练在开放领域、效率及可迁移性上的不足,提出MuDoK框架,实现多领域协同预训练与高效前缀提示调优,并构建开源基准KPI,实验显示其显著提升性能且具备通用性、高效性和可迁移性。

摘要:

知识图谱(KGs)以结构化三元组的形式为各种人工智能任务提供可靠的外部知识。知识图谱预训练(KGP)旨在大规模知识图谱上预训练神经网络,并提供统一接口以增强不同的下游任务,这是知识图谱管理、维护和应用的关键方向。现有工作往往专注于开放领域中的纯研究问题,或者由于真实场景中的数据安全和隐私问题而未开源。同时,现有研究尚未深入探索KGP模型的训练效率和可迁移性。为解决这些问题,我们提出了一个名为MuDoK的框架,以实现多领域协同预训练和高效的前缀提示调优,从而服务于推荐和文本理解等多样下游任务。我们的设计是一种即插即用的提示学习方法,可灵活适应不同的下游任务骨干网络。针对开源基准的缺乏,我们构建了一个新的多领域KGP基准,称为KPI,包含两个大规模知识图谱和六个不同的子领域任务,以评估我们的方法,并将其开源以供后续研究。我们基于构建的KPI基准,使用异构下游任务中的多种骨干模型评估了我们的方法。实验结果表明,我们的框架带来了显著的性能提升,同时具备通用性、高效性和可迁移性。

英文摘要:

Knowledge graphs (KGs) provide reliable external knowledge for a wide variety of AI tasks in the form of structured triples. Knowledge graph pre-training (KGP) aims to pre-train neural networks on large-scale KGs and provide unified interfaces to enhance different downstream tasks, which is a key direction for KG management, maintenance, and applications. Existing works often focus on purely research questions in open domains, or they are not open source due to data security and privacy in real scenarios. Meanwhile, existing studies have not explored the training efficiency and transferability of KGP models in depth. To address these problems, We propose a framework MuDoK to achieve multi-domain collaborative pre-training and efficient prefix prompt tuning to serve diverse downstream tasks like recommendation and text understanding. Our design is a plug-and-play prompt learning approach that can be flexibly adapted to different downstream task backbones. In response to the lack of open-source benchmarks, we constructed a new multi-domain KGP benchmark called KPI with two large-scale KGs and six different sub-domain tasks to evaluate our method and open-sourced it for subsequent research. We evaluated our approach based on constructed KPI benchmarks using diverse backbone models in heterogeneous downstream tasks. The experimental results show that our framework brings significant performance gains, along with its generality, efficiency, and transferability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑