Graph neural networks (GNNs) have exhibited remarkable performance under the assumption that test data comes from the same distribution of training data. However, in real-world scenarios, this assumption may not always be valid. Consequently, there is a growing focus on exploring the Out-of-Distribution (OOD) problem in the context of graphs. Most existing efforts have primarily concentrated on improving graph OOD generalization from two \textbf{model-agnostic} perspectives: data-driven methods and strategy-based learning. However, there has been limited attention dedicated to investigating the impact of well-known \textbf{GNN model architectures} on graph OOD generalization, which is orthogonal to existing research. In this work, we provide the first comprehensive investigation of OOD generalization on graphs from an architecture perspective, by examining the common building blocks of modern GNNs. Through extensive experiments, we reveal that both the graph self-attention mechanism and the decoupled architecture contribute positively to graph OOD generalization. In contrast, we observe that the linear classification layer tends to compromise graph OOD generalization capability. Furthermore, we provide in-depth theoretical insights and discussions to underpin these discoveries. These insights have empowered us to develop a novel GNN backbone model, DGAT, designed to harness the robust properties of both graph self-attention mechanism and the decoupled architecture. Extensive experimental results demonstrate the effectiveness of our model under graph OOD, exhibiting substantial and consistent enhancements across various training strategies.
LLM Enhancers for GNNs: An Analysis from the Perspective of Causal Mechanism Identification
Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);National Key Laboratory of Space Integrated Information System(空间信息集成系统国家重点实验室);University of Chinese Academy of Sciences(中国科学院大学);Tsinghua University(清华大学)
Survey on Graph Neural Network Acceleration: An Algorithmic Perspective
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Tsinghua University(清华大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Monash University(莫纳什大学);University of California, Santa Barbara(加州大学圣巴巴拉分校)
On the Computational Capability of Graph Neural Networks: A Circuit Complexity Bound Perspective
The University of Hong Kong(香港大学);University of Wisconsin-Madison(威斯康星大学麦迪逊分校);The Simons Institute for the Theory of Computing at UC Berkeley(加州大学伯克利分校西蒙斯理论计算研究所);UC Berkeley(加州大学伯克利分校)
Making Classic GNNs Strong Baselines Across Varying Homophily: A Smoothness-Generalization Perspective
Zhejiang Key Laboratory of Accessible Perception and Intelligent Systems, Zhejiang University(浙江可及感知与智能系统重点实验室,浙江大学);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);School of Software Technology, Zhejiang University(浙江大学软件学院);China University of Mining and Technology(中国矿业大学);Department of Computer Science, New York University Shanghai(纽约大学上海分校计算机科学系)
The Cyprus Institute(塞浦路斯研究所);National Institute of Science Education and Research(国家科学教育与研究研究所);Homi Bhabha National Institute(霍米·巴巴国家研究所);Microsoft Ltd.(微软有限公司)
Rethinking Causal Relationships Learning in Graph Neural Networks
Science and Technology on Integrated Information System Laboratory, Institute of Software Chinese Academy of Sciences(中国科学院软件研究所综合信息系统技术国家级重点实验室);University of Chinese Academy of Sciences(中国科学院大学);State Key Laboratory of Intelligent Game(智能博弈国家重点实验室);Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系)
School of Computer Science and Technology, China University of Mining and Technology(中国矿业大学计算机科学与技术学院);Mine Digitization Engineering Research Center, Ministry of Education of the People’s Republic of China(教育部矿山数字化工程研究中心)
Rethinking Graph Neural Networks for the Graph Coloring Problem
Carnegie Mellon University(卡内基梅隆大学);The Chinese University of Hong Kong(香港中文大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The University of Texas at Austin(德克萨斯大学奥斯汀分校)
School of Artificial Intelligence, Jilin University(吉林大学人工智能学院);Department of Computer Science, Rice University(莱斯大学计算机科学系);College of Computer Science and Technology, Jilin University(吉林大学计算机科学与技术学院)
National Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University, Wuhan, China(多媒体软件国家工程研究中心,计算机科学学院,武汉大学,武汉,中国)
Can Graph Neural Networks Learn to Solve MaxSAT Problem?
State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所计算机科学国家重点实验室);Laboratory of Parallel Software and Computational Science, Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所并行软件与计算科学实验室);University of Chinese Academy of Sciences(中国科学院大学)
Decision-focused Graph Neural Networks for Combinatorial Optimization
Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院);Guangzhou University(广州大学);Griffith University(格里菲斯大学);Hangzhou Yugu Technology(杭州宇谷科技);Zhejiang Lab(之江实验室)
Unleashing the potential of GNNs via Bi-directional Knowledge Transfer
Beijing Jiaotong University(北京交通大学);Beijing Key Laboratory of Advanced Information Science and Network Technology(北京先进信息科学与网络技术重点实验室);Qiyuan Lab(启元实验室);Beihang University(北京航空航天大学)
A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspective
The University of Hong Kong(香港大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Future Network of Intelligence Institute, CUHK-Shenzhen(香港中文大学(深圳)未来智联网络研究院);ShanghaiTech University(上海科技大学)
A Hitchhiker's Guide to Geometric GNNs for 3D Atomic Systems
Université Paris-Saclay(巴黎-萨克雷大学);CentraleSupélec(中央高等电力学院);Inria(法国国家信息与自动化研究所);Qualcomm AI Research(高通人工智能研究部);Mila(米拉计算科学研究所);University of Cambridge(剑桥大学);Université de Montréal(蒙特利尔大学);Intel Labs(英特尔实验室);Qualcomm Technologies, Inc.(高通技术公司);University of Oxford(牛津大学)