arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.01009cs.LG
收藏

通过动力系统视角看图神经网络中的过压缩问题

On Oversquashing in Graph Neural Networks Through the Lens of Dynamical Systems

University of Pisa(比萨大学);University of Cambridge(剑桥大学)

Alessio Gravina, Moshe Eliasof, Claudio Gallicchio, Davide Bacciu, Carola-Bibiane Schönlieb

AI总结:

本文利用动力系统的非耗散性,提出反对称GNN模型SWAN,以维持恒定信息流,缓解图神经网络中的过压缩问题,并在长程交互基准上验证其有效性。

摘要:

消息传递神经网络中的一个常见问题是过压缩——即在远距离节点之间促进有效信息流动的能力有限。过压缩被归因于随着节点距离增加,信息传输呈指数衰减。本文引入了一种新颖的视角来解决过压缩问题,利用全局和局部非耗散性的动力系统性质,这些性质能够维持恒定的信息流速率。我们提出了SWAN,一个在空间域和权重域均具有反对称性的独特参数化GNN模型,作为实现非耗散性的一种手段。我们的理论分析断言,通过实现这些性质,SWAN在远距离传输信息方面提供了增强的能力。在强调长程交互的合成和真实世界基准上的实证评估验证了对SWAN的理论理解及其缓解过压缩的能力。

英文摘要:

A common problem in Message-Passing Neural Networks is oversquashing -- the limited ability to facilitate effective information flow between distant nodes. Oversquashing is attributed to the exponential decay in information transmission as node distances increase. This paper introduces a novel perspective to address oversquashing, leveraging dynamical systems properties of global and local non-dissipativity, that enable the maintenance of a constant information flow rate. We present SWAN, a uniquely parameterized GNN model with antisymmetry both in space and weight domains, as a means to obtain non-dissipativity. Our theoretical analysis asserts that by implementing these properties, SWAN offers an enhanced ability to transmit information over extended distances. Empirical evaluations on synthetic and real-world benchmarks that emphasize long-range interactions validate the theoretical understanding of SWAN, and its ability to mitigate oversquashing.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑