arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.06970cs.LG
收藏

Flood and Echo Net:算法对齐且具备泛化能力的图神经网络

Flood and Echo Net: Algorithmically Aligned GNNs that Generalize

ETH Zurich(苏黎世联邦理工学院);IIIT Bangalore(印度信息技术研究所班加罗尔分校)

Joël Mathys, Florian Grötschla, Kalyan Varma Nadimpalli, Roger Wattenhofer

AI总结:

提出 Flood and Echo Net,通过模拟分布式算法的洪泛与回声阶段实现稀疏并行激活,克服传统 MPNNs 的 1-WL 限制,在提升效率与表达能力的同时增强了对不同规模图的泛化能力。

摘要:

大多数图神经网络遵循标准的消息传递框架,其中在每一步,所有节点同时相互通信。我们希望通过将计算更紧密地对齐到分布式算法的执行来挑战这一范式,并提出 Flood and Echo Net。Flood and Echo Net 的一轮由一个源节点、一个洪泛阶段以及随后的回声阶段组成。首先,在洪泛期间,消息从源节点发出并向外传播至整个图。然后,在回声期间,消息流反转,消息被发送回源节点。由于节点仅在收到消息时被稀疏激活,这导致了遍历图的波状激活模式。通过这些稀疏但并行的激活,该网络比受限于 1-WL 测试的传统 MPNNs 更具表达能力,并且在消息复杂度方面可证明更高效。此外,该机制在跨不同大小图进行泛化方面的固有能力,使其成为算法学习任务的实用架构。我们在各种合成任务和 SALSA-CLRS 基准上测试了 Flood and Echo Net,发现执行的算法对齐改善了对更大图尺寸的泛化能力。

英文摘要:

Most Graph Neural Networks follow the standard message-passing framework where, in each step, all nodes simultaneously communicate with each other. We want to challenge this paradigm by aligning the computation more closely to the execution of distributed algorithms and propose the Flood and Echo Net. A single round of a Flood and Echo Net consists of an origin node and a flooding phase followed by an echo phase. First, during the flooding, messages are sent from the origin and propagated outwards throughout the entire graph. Then, during the echo, the message flow reverses and messages are sent back towards the origin. As nodes are only sparsely activated upon receiving a message, this leads to a wave-like activation pattern that traverses the graph. Through these sparse but parallel activations, the Net becomes more expressive than traditional MPNNs which are limited by the 1-WL test and also is provably more efficient in terms of message complexity. Moreover, the mechanism's inherent ability to generalize across graphs of varying sizes positions it as a practical architecture for the task of algorithmic learning. We test the Flood and Echo Net on a variety of synthetic tasks and the SALSA-CLRS benchmark and find that the algorithmic alignment of the execution improves generalization to larger graph sizes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑