全局通信还是图专用记忆？

中文摘要 AI 辅助

可扩展图变换器通常在直推式设置下，针对静态大型图进行训练和评估。许多可扩展图变换器组件可以被表述为恒定大小的共享记忆，类似于虚拟节点，提供关于整个图的压缩信息。这些模型在语言模型和其他领域中的对应物之所以合理，是因为输入会变化，而该机制学习压缩关于输入的一些有用信息。然而，在单个固定图上的直推式学习中，任何共享记忆在测试时都可以被视为常量。这引出了一个问题：在这种静态设置中，共享记忆究竟起到了什么作用。我们提供了初步证据表明，直接优化共享记忆的表现与全局通信方法相似，因此普通的局部消息传递模型可以在其权重中嵌入类似的信息。因此，这些设置可能不适合评估图神经网络中的全局通信。

英文摘要

Scalable Graph Transformers are commonly trained and evaluated on static large graphs in a transductive setup. Many scalable Graph Transformer components can be formulated as a constant-size shared memory, similar to virtual nodes, providing compressed information about the whole graph. The counterpart of these models in language models and other domains is justified as the input changes, and this mechanism learns to compress some useful information about the input. In transductive learning on a single fixed graph, however, any shared memory can be seen as a constant at test time. This raises the question of what exactly this shared memory does in this static setup. We give preliminary evidence that optimizing a shared memory directly performs similarly to global communication methods, and so normal local message-passing models can embed similar information in their weights. Thus, these settings may be a poor fit for evaluating global communication in graph neural networks.

发表机构

• University of British Columbia（不列颠哥伦比亚大学）

机构由 AI 辅助整理，请以论文原文为准。