arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.08228cs.LGcs.AI
收藏

探究GNN的分布外泛化:一种架构视角

Investigating Out-of-Distribution Generalization of GNNs: An Architecture Perspective

Michigan State University(密歇根州立大学);Emory University(埃默里大学)

Kai Guo, Hongzhi Wen, Wei Jin, Yaming Guo, Jiliang Tang, Yi Chang

AI总结:

本研究首次从架构视角探究图神经网络的分布外泛化问题,发现图自注意力机制和解耦架构能提升泛化能力,而线性分类层会削弱该能力,并据此提出了新型主干模型DGAT。

摘要:

图神经网络(GNNs)在测试数据与训练数据同分布的假设下展现出了卓越的性能。然而,在现实场景中,该假设并不总是成立。因此,图背景下分布外(OOD)问题的探索日益受到关注。现有的大多数努力主要集中在从两个与模型无关的视角——数据驱动方法和基于策略的学习——来改善图的OOD泛化。然而,对于著名的GNN模型架构对图OOD泛化的影响,关注度有限,且该方向与现有研究正交。在本研究中,我们通过检验现代GNNs的常见构建模块,首次从架构视角对图上的OOD泛化进行了全面探究。通过大量实验,我们揭示了图自注意力机制和解耦架构均对图OOD泛化有积极贡献。相反,我们观察到线性分类层倾向于削弱图的OOD泛化能力。此外,我们提供了深入的理论见解和讨论来支撑这些发现。这些见解使我们能够开发一种新型GNN主干模型DGAT,旨在利用图自注意力机制和解耦架构的稳健特性。大量实验结果表明,我们的模型在图OOD下具有有效性,在各种训练策略中展现出显著且一致的提升。

英文摘要:

Graph neural networks (GNNs) have exhibited remarkable performance under the assumption that test data comes from the same distribution of training data. However, in real-world scenarios, this assumption may not always be valid. Consequently, there is a growing focus on exploring the Out-of-Distribution (OOD) problem in the context of graphs. Most existing efforts have primarily concentrated on improving graph OOD generalization from two \textbf{model-agnostic} perspectives: data-driven methods and strategy-based learning. However, there has been limited attention dedicated to investigating the impact of well-known \textbf{GNN model architectures} on graph OOD generalization, which is orthogonal to existing research. In this work, we provide the first comprehensive investigation of OOD generalization on graphs from an architecture perspective, by examining the common building blocks of modern GNNs. Through extensive experiments, we reveal that both the graph self-attention mechanism and the decoupled architecture contribute positively to graph OOD generalization. In contrast, we observe that the linear classification layer tends to compromise graph OOD generalization capability. Furthermore, we provide in-depth theoretical insights and discussions to underpin these discoveries. These insights have empowered us to develop a novel GNN backbone model, DGAT, designed to harness the robust properties of both graph self-attention mechanism and the decoupled architecture. Extensive experimental results demonstrate the effectiveness of our model under graph OOD, exhibiting substantial and consistent enhancements across various training strategies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑