arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2105.07944cs.LGcs.AI
收藏

TCL:基于Transformer的对比学习动态图建模

TCL: Transformer-based Dynamic Graph Modelling via Contrastive Learning

East China Normal University(华东师范大学);Damo Academy, Alibaba Group(阿里巴巴集团达摩院);Harvard University(哈佛大学);Ant Group(蚂蚁集团);Gatech(佐治亚理工学院)

Lu Wang, Xiaofu Chang, Shuang Li, Yunfei Chu, Hui Li, Wei Zhang, Xiaofeng He, Le Song, Jingren Zhou, Hongxia Yang

摘要:

动态图建模由于其在推荐系统、金融交易、社交网络等领域的广泛应用而受到广泛关注。虽然近年来已经提出了许多工作动态图建模,有效的和可扩展的模型还有待开发。在本文中,我们提出了一种新的图神经网络的方法,称为TCL,它处理动态演变的图形在连续时间的方式,并使有效的动态节点表示学习,捕捉时间和拓扑信息。从技术上讲,我们的模型包含三个新的方面。首先,我们将普通的Transformer推广到时态图学习场景,并设计了一个图拓扑感知的Transformer。其次,顶部的建议图Transformer,我们引入了一个两个流编码器,分别提取表示与两个相互作用的节点,然后利用共同关注的Transformer模型在语义层面上的相互依赖关系。最后,我们受到最近开发的对比学习的启发,提出通过最大化两个未来交互节点的预测表示之间的互信息(MI)来优化我们的模型。受益于此,我们的动态表示可以保持高层次(或全球)语义的互动,从而是强大的嘈杂的互动。据我们所知,这是首次尝试将对比学习应用于动态图上的表示学习。我们在四个基准数据集上评估了我们的模型,实验结果证明了我们的模型的优越性。

英文摘要:

Dynamic graph modeling has recently attracted much attention due to its extensive applications in many real-world scenarios, such as recommendation systems, financial transactions, and social networks. Although many works have been proposed for dynamic graph modeling in recent years, effective and scalable models are yet to be developed. In this paper, we propose a novel graph neural network approach, called TCL, which deals with the dynamically-evolving graph in a continuous-time fashion and enables effective dynamic node representation learning that captures both the temporal and topology information. Technically, our model contains three novel aspects. First, we generalize the vanilla Transformer to temporal graph learning scenarios and design a graph-topology-aware transformer. Secondly, on top of the proposed graph transformer, we introduce a two-stream encoder that separately extracts representations from temporal neighborhoods associated with the two interaction nodes and then utilizes a co-attentional transformer to model inter-dependencies at a semantic level. Lastly, we are inspired by the recently developed contrastive learning and propose to optimize our model by maximizing mutual information (MI) between the predictive representations of two future interaction nodes. Benefiting from this, our dynamic representations can preserve high-level (or global) semantics about interactions and thus is robust to noisy interactions. To the best of our knowledge, this is the first attempt to apply contrastive learning to representation learning on dynamic graphs. We evaluate our model on four benchmark datasets for interaction prediction and experiment results demonstrate the superiority of our model.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑