arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.04713cs.LGcs.AI
收藏

DyGMamba:利用状态空间模型在连续时间动态图上高效建模长期时间依赖

DyGMamba: Efficiently Modeling Long-Term Temporal Dependency on Continuous-Time Dynamic Graphs with State Space Models

University of Cambridge(剑桥大学);Technical University of Munich(慕尼黑技术大学);University of Oxford(牛津大学);University of Stuttgart(斯图加特大学);LMU Munich(慕尼黑大学);Munich Center for Machine Learning(慕尼黑机器学习中心)

Zifeng Ding, Yifeng Li, Yuan He, Antonio Norelli, Jingcheng Wu, Volker Tresp, Michael Bronstein, Yunpu Ma

AI总结:

本文提出DyGMamba模型,通过节点级和时间级状态空间模型编码连续时间动态图中的长期交互历史并动态选择关键时间信息,在动态链接预测任务上达到最先进水平且计算高效。

摘要:

学习连续时间动态图(CTDGs)的有用表示具有挑战性,因为需要同时跨越较长的节点交互历史并把握细微的时间细节。特别是出现了两个问题:(1)编码更长的历史需要更多的计算资源,因此CTDG模型保持低计算复杂度以确保效率至关重要;(2)同时,需要更强大的模型来识别和选择更长历史所提供的扩展上下文中最关键的时间信息。为了解决这些问题,我们提出了一种名为DyGMamba的CTDG表示学习模型,该模型源自流行的Mamba状态空间模型(SSM)。DyGMamba首先利用节点级SSM来编码历史节点交互序列。然后采用另一个时间级SSM来利用历史图中隐藏的时间模式,其输出用于从交互历史中动态选择关键信息。我们在动态链接预测任务上对DyGMamba进行了实验验证。结果表明,我们的模型在大多数情况下达到了最先进的水平。DyGMamba在计算资源方面也保持了高效率,使得在有限的计算预算下捕获长期时间依赖成为可能。

英文摘要:

Learning useful representations for continuous-time dynamic graphs (CTDGs) is challenging, due to the concurrent need to span long node interaction histories and grasp nuanced temporal details. In particular, two problems emerge: (1) Encoding longer histories requires more computational resources, making it crucial for CTDG models to maintain low computational complexity to ensure efficiency; (2) Meanwhile, more powerful models are needed to identify and select the most critical temporal information within the extended context provided by longer histories. To address these problems, we propose a CTDG representation learning model named DyGMamba, originating from the popular Mamba state space model (SSM). DyGMamba first leverages a node-level SSM to encode the sequence of historical node interactions. Another time-level SSM is then employed to exploit the temporal patterns hidden in the historical graph, where its output is used to dynamically select the critical information from the interaction history. We validate DyGMamba experimentally on the dynamic link prediction task. The results show that our model achieves state-of-the-art in most cases. DyGMamba also maintains high efficiency in terms of computational resources, making it possible to capture long temporal dependencies with a limited computation budget.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑