arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.04562cs.CLcs.AI
收藏

迈向知识图谱推理的基础模型

Towards Foundation Models for Knowledge Graph Reasoning

Intel AI Lab(英特尔人工智能实验室);Mila(米拉计算研究所);University of Montréal(蒙特利尔大学);HEC Montréal(蒙特利尔高等商学院)

Mikhail Galkin, Xinyu Yuan, Hesham Mostafa, Jian Tang, Zhaocheng Zhu

AI总结:

本文提出 ULTRA,通过以关系交互为条件学习通用可迁移图表示,使单个预训练模型可零样本泛化并微调到任意知识图谱,在57个KG上取得可比或优于专用基线的链接预测效果。

摘要:

语言和视觉领域的基础模型之所以能够对任意文本和视觉输入执行推理,得益于可迁移表示,例如语言中的 token 词汇表。知识图谱(KG)具有通常互不重叠的不同实体和关系词汇表。在 KG 上设计基础模型的关键挑战,是学习这种可迁移表示,从而能够在具有任意实体和关系词汇表的任意图上进行推理。在本工作中,我们朝着此类基础模型迈出一步,并提出 ULTRA,这是一种用于学习通用且可迁移的图表示的方法。ULTRA 将关系表示构建为以关系之间的交互为条件的函数。这种条件化策略使预训练的 ULTRA 模型能够归纳泛化到具有任意关系词汇表的任何未见 KG,并可在任意图上进行微调。通过在 57 个不同 KG 上开展链接预测实验,我们发现,单个预训练 ULTRA 模型在各种规模的未见图上的零样本归纳推理性能,往往与在特定图上训练的强基线相当,甚至更优。微调还会进一步提升性能。

英文摘要:

Foundation models in language and vision have the ability to run inference on any textual and visual inputs thanks to the transferable representations such as a vocabulary of tokens in language. Knowledge graphs (KGs) have different entity and relation vocabularies that generally do not overlap. The key challenge of designing foundation models on KGs is to learn such transferable representations that enable inference on any graph with arbitrary entity and relation vocabularies. In this work, we make a step towards such foundation models and present ULTRA, an approach for learning universal and transferable graph representations. ULTRA builds relational representations as a function conditioned on their interactions. Such a conditioning strategy allows a pre-trained ULTRA model to inductively generalize to any unseen KG with any relation vocabulary and to be fine-tuned on any graph. Conducting link prediction experiments on 57 different KGs, we find that the zero-shot inductive inference performance of a single pre-trained ULTRA model on unseen graphs of various sizes is often on par or better than strong baselines trained on specific graphs. Fine-tuning further boosts the performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑