arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2104.01481cs.LG
收藏

我们需要各向异性图神经网络吗?

Do We Need Anisotropic Graph Neural Networks?

University of Cambridge(剑桥大学);Invenia Labs(Invenia实验室);Samsung AI Center(三星人工智能中心)

Shyam A. Tailor, Felix L. Opolka, Pietro Liò, Nicholas D. Lane

摘要:

图神经网络(GNN)社区的共同智慧表明,各向异性模型-其中节点之间发送的消息是源节点和目标节点的函数-需要实现最先进的性能。迄今为止的基准测试表明,这些模型的性能优于可比的各向同性模型-其中消息仅是源节点的函数。在这项工作中,我们提供了挑战这种叙述的经验证据:我们提出了一个各向同性的GNN,我们称之为高效图卷积(EGC),它始终优于可比的各向异性模型,包括流行的GAT或PNA架构,通过使用空间变化的自适应滤波器。除了为GNN社区提出重要问题外,我们的工作对效率具有重要的现实意义。EGC实现了更高的模型精度,更低的内存消耗和延迟,以及适合加速器实现的特性,同时是现有架构的直接替代品。作为各向同性模型,它需要与图中的顶点数($\mathcal{O}(V)$)成比例的内存;相反,各向异性模型需要与边数($\mathcal{O}(E)$)成比例的内存。我们证明了EGC在6个大型和多样化的基准数据集上优于现有方法,并通过讨论我们的工作为社区提出的问题得出结论。我们实验的代码和预训练模型在https://github.com/shyam196/egc上提供。

英文摘要:

Common wisdom in the graph neural network (GNN) community dictates that anisotropic models -- in which messages sent between nodes are a function of both the source and target node -- are required to achieve state-of-the-art performance. Benchmarks to date have demonstrated that these models perform better than comparable isotropic models -- where messages are a function of the source node only. In this work we provide empirical evidence challenging this narrative: we propose an isotropic GNN, which we call Efficient Graph Convolution (EGC), that consistently outperforms comparable anisotropic models, including the popular GAT or PNA architectures by using spatially-varying adaptive filters. In addition to raising important questions for the GNN community, our work has significant real-world implications for efficiency. EGC achieves higher model accuracy, with lower memory consumption and latency, along with characteristics suited to accelerator implementation, while being a drop-in replacement for existing architectures. As an isotropic model, it requires memory proportional to the number of vertices in the graph ($\mathcal{O}(V)$); in contrast, anisotropic models require memory proportional to the number of edges ($\mathcal{O}(E)$). We demonstrate that EGC outperforms existing approaches across 6 large and diverse benchmark datasets, and conclude by discussing questions that our work raise for the community going forward. Code and pretrained models for our experiments are provided at https://github.com/shyam196/egc.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑