arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2009.14332cs.LGstat.ML
收藏

多跳注意力图神经网络

Multi-hop Attention Graph Neural Network

JD AI Research(京东人工智能研究院);Stanford University(斯坦福大学)

Guangtao Wang, Rex Ying, Jing Huang, Jure Leskovec

摘要:

图神经网络的自注意机制在许多图表示学习任务上具有最先进的性能。目前,在每一层,注意力都是在连接的节点对之间计算的,并且仅仅取决于两个节点的表示。然而,这种注意力机制不考虑不直接连接但提供重要网络上下文的节点。在这里,我们提出了多跳注意力图神经网络(MAGNA),一个原则性的方法,将多跳上下文信息到每一层的注意力计算。MAGNA将注意力分数扩散到整个网络,从而增加GNN每一层的感受野。与以前的方法不同,MAGNA使用注意力值的扩散先验,以有效地解释断开连接的节点对之间的所有路径。我们在理论和实验中证明,MAGNA捕获大规模的结构信息,在每一层,并具有低通效应,消除噪声的高频信息从图形数据。节点分类以及知识图谱完成基准测试的实验结果表明,MAGNA实现了最先进的结果:MAGNA实现了高达5.7%的相对错误减少比以前的最先进的Cora,Citeseer和Pubmed。MAGNA还在大规模Open Graph Benchmark数据集上获得了最佳性能。在知识图谱完成方面,MAGNA在WN 18 RR和FB 15 k-237上的四个不同性能指标上都达到了最先进的水平。

英文摘要:

Self-attention mechanism in graph neural networks (GNNs) led to state-of-the-art performance on many graph representation learning tasks. Currently, at every layer, attention is computed between connected pairs of nodes and depends solely on the representation of the two nodes. However, such attention mechanism does not account for nodes that are not directly connected but provide important network context. Here we propose Multi-hop Attention Graph Neural Network (MAGNA), a principled way to incorporate multi-hop context information into every layer of attention computation. MAGNA diffuses the attention scores across the network, which increases the receptive field for every layer of the GNN. Unlike previous approaches, MAGNA uses a diffusion prior on attention values, to efficiently account for all paths between the pair of disconnected nodes. We demonstrate in theory and experiments that MAGNA captures large-scale structural information in every layer, and has a low-pass effect that eliminates noisy high-frequency information from graph data. Experimental results on node classification as well as the knowledge graph completion benchmarks show that MAGNA achieves state-of-the-art results: MAGNA achieves up to 5.7 percent relative error reduction over the previous state-of-the-art on Cora, Citeseer, and Pubmed. MAGNA also obtains the best performance on a large-scale Open Graph Benchmark dataset. On knowledge graph completion MAGNA advances state-of-the-art on WN18RR and FB15k-237 across four different performance metrics.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑