arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.11636cs.LGcs.AI
收藏

我们真的需要为时序网络构建复杂的模型架构吗?

Do We Really Need Complicated Model Architectures For Temporal Networks?

Penn State(宾夕法尼亚州立大学);Meta;University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)

Weilin Cong, Si Zhang, Jian Kang, Baichuan Yuan, Hao Wu, Xin Zhou, Hanghang Tong, Mehrdad Mahdavi

摘要:

递归神经网络(RNN)和自注意机制(SAM)是用于时间图学习的时空信息提取方法。有趣的是,我们发现,尽管RNN和SAM都可以带来良好的性能,但在实践中,它们都不总是必要的。在本文中,我们提出了GraphMixer,这是一个概念上和技术上都很简单的架构,由三个组件组成:(1)仅基于多层感知器(MLP)来总结来自时间链路的信息的链路编码器,(2)仅基于邻居均值池来总结节点信息的节点编码器,以及(3)基于MLP的链路分类器,其基于编码器的输出执行链接预测。尽管它很简单,但GraphMixer在时间链接预测基准测试中表现出色,收敛速度更快,泛化性能更好。这些结果促使我们重新思考更简单的模型架构的重要性。

英文摘要:

Recurrent neural network (RNN) and self-attention mechanism (SAM) are the de facto methods to extract spatial-temporal information for temporal graph learning. Interestingly, we found that although both RNN and SAM could lead to a good performance, in practice neither of them is always necessary. In this paper, we propose GraphMixer, a conceptually and technically simple architecture that consists of three components: (1) a link-encoder that is only based on multi-layer perceptrons (MLP) to summarize the information from temporal links, (2) a node-encoder that is only based on neighbor mean-pooling to summarize node information, and (3) an MLP-based link classifier that performs link prediction based on the outputs of the encoders. Despite its simplicity, GraphMixer attains an outstanding performance on temporal link prediction benchmarks with faster convergence and better generalization performance. These results motivate us to rethink the importance of simpler model architecture.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑