Message passing graph neural networks (GNNs) are a popular learning architectures for graph-structured data. However, one problem GNNs experience is oversquashing, where a GNN has difficulty sending information between distant nodes. Understanding and mitigating oversquashing has recently received significant attention from the research community. In this paper, we continue this line of work by analyzing oversquashing through the lens of the effective resistance between nodes in the input graph. Effective resistance intuitively captures the ``strength'' of connection between two nodes by paths in the graph, and has a rich literature spanning many areas of graph theory. We propose to use total effective resistance as a bound of the total amount of oversquashing in a graph and provide theoretical justification for its use. We further develop an algorithm to identify edges to be added to an input graph to minimize the total effective resistance, thereby alleviating oversquashing. We provide empirical evidence of the effectiveness of our total effective resistance based rewiring strategies for improving the performance of GNNs.
Beyond Oversquashing: Understanding Signal Propagation in GNNs Via Observables
Technion - Israel Institute of Technology, Faculty of Mathematics(技术离子-以色列理工学院,数学系);Technical University of Munich, School of Computation, Information and Technology(慕尼黑技术大学,计算、信息与技术学院);Munich Center for Machine Learning(慕尼黑机器学习中心)
Cross-attentive Cohesive Subgraph Embedding to Mitigate Oversquashing in GNNs
Department of Computer Science, Georgia State University(佐治亚州立大学计算机科学系);Department of Data Science and Analytics, Kennesaw State University(肯尼索州立大学数据科学与分析系)
Sketch-Augmented Features Improve Learning Long-Range Dependencies in Graph Neural Networks
Department of Computer Science, University of Chicago(芝加哥大学计算机科学系);Leadership Computing Facility, Argonne National Laboratory(阿贡国家实验室领导计算设施);Department of Statistics, University of Chicago(芝加哥大学统计系);NSF-Simons National Institute for Theory and Mathematics in Biology(国家理论与生物数学研究所)
本文提出将随机化全局嵌入(Sketched Random Features)注入标准图神经网络,以缓解过度挤压、过度平滑和表达力受限问题,从而高效捕获长程依赖并在真实图任务上持续提升性能。
SCNode: Spatial and Contextual Coordinates for Graph Representation Learning
Department of Mathematical Science(数学科学系);The University of Texas at Dallas(德克萨斯大学达拉斯分校);Department of Mathematics(数学系);Florida State University(佛罗里达州立大学);Department of Computer Science(计算机科学系);AI Initiative(人工智能计划);University of Central Florida(中央佛罗里达大学)
Probing Graph Neural Network Activation Patterns Through Graph Topology
Department of Computer Science, University of Geneva, Geneva, Switzerland(日内瓦大学计算机科学系);Research Institute for Statistics and Information Science, Geneva, Switzerland(日内瓦统计与信息科学研究所);School of Engineering and Applied Sciences, Harvard University(哈佛大学工程与应用科学学院)
A Polynomial Architecture-Attribution Co-Design Framework for Exact Aumann-Shapley Attribution in GNNs
Institute of Artificial Intelligence Innovation and Industry, Fudan University(复旦大学人工智能创新与产业研究院);Shanghai Academy of AI for Science(上海人工智能科学研究院);Human Phenome Institute, Fudan University(复旦大学人类表型组研究院);School of Information and Communication Engineering, Communication University of China(中国传媒大学信息与通信工程学院)
RAwR: Role-Aware Rewiring via Approximate Equitable Partition
Department of Excellence L’EMbeDS, Sant’Anna School of Advanced Studies, Pisa, Italy(卓越部门L’EMbeDS,圣安娜高级研究学院,意大利比萨);Department of Computer Science, University of Pisa, Italy(计算机科学系,意大利比萨大学);Department of Statistics, The Pennsylvania State University, USA(统计系,美国宾夕法尼亚州立大学);Huck Institutes of the Life Sciences, The Pennsylvania State University, USA(生命科学学院Huck研究所,美国宾夕法尼亚州立大学);IMT School for Advanced Studies, Lucca, Italy(IMT高级研究学院,意大利卢卡);Computational Network Science, RWTH Aachen University, Aachen, Germany(计算网络科学,德国亚琛工业大学,亚琛);DTU Technical University of Denmark, Lyngby, Denmark(丹麦技术大学DTU,Lyngby,丹麦)
A Geometric Perspective on the Difficulties of Learning GNN-based SAT Solvers
Department of Computing Sciences, Bocconi University, Milan, Italy(计算机科学系,博科尼大学,米兰,意大利);Bocconi Institute for Data Science and Analytics (BIDSA)(博科尼数据科学与分析研究所)
On the Computational Capability of Graph Neural Networks: A Circuit Complexity Bound Perspective
The University of Hong Kong(香港大学);University of Wisconsin-Madison(威斯康星大学麦迪逊分校);The Simons Institute for the Theory of Computing at UC Berkeley(加州大学伯克利分校西蒙斯理论计算研究所);UC Berkeley(加州大学伯克利分校)
ADAPT: Lightweight, Long-Range Machine Learning Force Fields Without Graphs
Department of Computer Science, Rice University(计算机科学系,里士大学);Rice Advanced Materials Institute, Rice University(里士大学先进材料研究所);Department of Materials Science and Nanoengineering, Rice University(材料科学与纳米工程系,里士大学);Thayer School of Engineering, Dartmouth College(达特茅斯学院泰勒工程学院);Department of Computer Sciences, University of Wisconsin–Madison(威斯康星大学麦迪逊分校计算机科学系)
Beyond Edge Deletion: A Comprehensive Approach to Counterfactual Explanation in Graph Neural Networks
Sapienza University of Rome Rome Italy;ISTC-CNR \& Sapienza University of Rome Rome Italy;Technical University of Munich Munich Germany;Sapienza University of Rome;ISTC-CNR \& Sapienza University of Rome;Technical University of Munich
Partition-wise Graph Filtering: A Unified Perspective Through the Lens of Graph Coarsening
Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);University of Chinese Academy of Sciences(中国科学院大学);Hong Kong Baptist University(香港 Baptist 大学)
Link Stealing Attacks Against Inductive Graph Neural Networks
CISPA Helmholtz Center for Information Security(CISPA亥姆霍兹信息安全研究中心);Hong Kong University of Science and Technology(香港科技大学);University of Birmingham(伯明翰大学);University of Lausanne(洛桑大学);Duke University(杜克大学)
Massachusetts Institute of Technology (MIT)(麻省理工学院);University of Maryland(马里兰大学);Institute for Advanced Study (IAS)(普林斯顿高等研究院);National Institute of Informatics (NII)(国立信息学研究所)
Rethinking Graph Neural Networks for Anomaly Detection
Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Hong Kong University of Science and Technology(香港科技大学);Stanford University(斯坦福大学)