自从Gori et al.（2005）和Scarselli et al.（2008）提出图神经网络（GNN）以来，训练GNN的主要问题之一是它们难以在图中遥远的节点之间传播信息。我们对这个问题提出了一个新的解释：GNN在聚合长路径上的消息时容易受到瓶颈的影响。这个瓶颈会导致指数增长的信息过度压缩到固定大小的向量中。因此，GNN无法传播来自远程节点的消息，并且在预测任务依赖于远程交互时表现不佳。在本文中，我们强调了GNN中的过度挤压的固有问题：我们证明了瓶颈阻碍了流行的GNN拟合训练数据中的长距离信号;我们进一步表明，平均吸收传入边缘的GNN，如GCN和GIN，比GAT和GGNN更容易受到过度挤压的影响;最后，我们表明，以前的工作，广泛调整GNN模型的长期问题，遭受过度挤压，打破瓶颈，提高了他们的国家的最先进的结果，没有任何调整或额外的权重。我们的代码可在https://github.com/tech-srl/bottleneck/上获得。
英文摘要：
Since the proposal of the graph neural network (GNN) by Gori et al. (2005) and Scarselli et al. (2008), one of the major problems in training GNNs was their struggle to propagate information between distant nodes in the graph. We propose a new explanation for this problem: GNNs are susceptible to a bottleneck when aggregating messages across a long path. This bottleneck causes the over-squashing of exponentially growing information into fixed-size vectors. As a result, GNNs fail to propagate messages originating from distant nodes and perform poorly when the prediction task depends on long-range interaction. In this paper, we highlight the inherent problem of over-squashing in GNNs: we demonstrate that the bottleneck hinders popular GNNs from fitting long-range signals in the training data; we further show that GNNs that absorb incoming edges equally, such as GCN and GIN, are more susceptible to over-squashing than GAT and GGNN; finally, we show that prior work, which extensively tuned GNN models of long-range problems, suffers from over-squashing, and that breaking the bottleneck improves their state-of-the-art results without any tuning or additional weights. Our code is available at https://github.com/tech-srl/bottleneck/ .
Distributed Information Bottleneck Theory for Multi-Modal Task-Aware Semantic Communication
School of Electronic Information and Communications, Huazhong University of Science and Technology（电子信息与通信学院，华中科技大学）；Peng Cheng Laboratory（鹏城实验室）；Pazhou Laboratory (Huangpu)（琶洲实验室（黄埔））；School of Mechanical Engineering and Electronic Information, China University of Geosciences（机械工程与电子信息学院，中国地质大学）；State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications（网络与交换技术国家重点实验室，北京邮电大学）
