arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2405.00352cs.AI
收藏

基于Transformer的时间知识图谱事件演化链学习推理

Transformer-based Reasoning for Learning Evolutionary Chain of Events on Temporal Knowledge Graph

University of Science and Technology Beijing(北京科技大学)

Zhiyu Fang, Shuai-Long Lei, Xiaobin Zhu, Chun Yang, Shi-Xue Zhang, Xu-Cheng Yin, Jingyan Qin

AI总结:

本文提出基于Transformer的ECEformer模型,通过按时间顺序构建事件演化链并学习四元组内外部统一表示,提升时间知识图谱推理性能。

摘要:

时间知识图谱(Temporal Knowledge Graph, TKG)推理通常涉及沿时间线补全缺失的事实元素。尽管现有方法能够通过整合时间信息为四元组中的每个事实元素学习良好的嵌入,但它们往往无法推断时间事实的演化。这主要是因为:(1)未能充分探索单个四元组内部的结构和语义关系;(2)未能充分学习不同四元组之间上下文和时间相关性的统一表示。为克服这些局限,我们提出了一种新颖的基于Transformer的TKG推理模型(称为ECEformer),用于学习事件演化链(Evolutionary Chain of Events, ECE)。具体而言,我们按时间顺序展开实体节点的邻域子图,形成事件演化链作为模型输入。随后,我们利用Transformer编码器学习ECE的四元组内部嵌入。接着,我们基于多层感知机(MLP)构建了一个混合上下文推理模块,以学习ECE的四元组间统一表示,同时完成时间知识推理。此外,为增强事件的时效性,我们设计了一个额外的时间预测任务,以在学习到的统一表示中补全有效的时间信息。在六个基准数据集上的大量实验验证了我们方法的最先进性能和有效性。

英文摘要:

Temporal Knowledge Graph (TKG) reasoning often involves completing missing factual elements along the timeline. Although existing methods can learn good embeddings for each factual element in quadruples by integrating temporal information, they often fail to infer the evolution of temporal facts. This is mainly because of (1) insufficiently exploring the internal structure and semantic relationships within individual quadruples and (2) inadequately learning a unified representation of the contextual and temporal correlations among different quadruples. To overcome these limitations, we propose a novel Transformer-based reasoning model (dubbed ECEformer) for TKG to learn the Evolutionary Chain of Events (ECE). Specifically, we unfold the neighborhood subgraph of an entity node in chronological order, forming an evolutionary chain of events as the input for our model. Subsequently, we utilize a Transformer encoder to learn the embeddings of intra-quadruples for ECE. We then craft a mixed-context reasoning module based on the multi-layer perceptron (MLP) to learn the unified representations of inter-quadruples for ECE while accomplishing temporal knowledge reasoning. In addition, to enhance the timeliness of the events, we devise an additional time prediction task to complete effective temporal information within the learned unified representation. Extensive experiments on six benchmark datasets verify the state-of-the-art performance and the effectiveness of our method.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑