arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.00613cs.LG
收藏

分散图形注意力

Diffusing Graph Attention

Tel-Aviv University(特拉维夫大学);Technion(以色列理工学院)

Daniel Glickman, Eran Yahav

摘要:

图上机器学习的主要范例使用消息传递图神经网络(MP-GNN),其中节点表示通过聚合其局部邻域中的信息来更新。最近,已经有越来越多的尝试使Transformer架构适应于图形,以努力解决MP-GNN的一些已知限制。设计图Transformers的一个挑战性方面是将任意图结构集成到体系结构中。我们提出了图扩散器(GD)来解决这一挑战。GD学习提取图中遥远节点之间的结构和位置关系,然后使用这些关系来引导Transformer的注意力和节点表示。我们展示了现有的GNN和Graph Transformers很难捕捉远程交互,以及Graph Diffuser如何在承认直观可视化的同时做到这一点。在八个基准测试上的实验表明,Graph Diffuser是一个非常有竞争力的模型,在不同的领域中表现优于最先进的模型。

英文摘要:

The dominant paradigm for machine learning on graphs uses Message Passing Graph Neural Networks (MP-GNNs), in which node representations are updated by aggregating information in their local neighborhood. Recently, there have been increasingly more attempts to adapt the Transformer architecture to graphs in an effort to solve some known limitations of MP-GNN. A challenging aspect of designing Graph Transformers is integrating the arbitrary graph structure into the architecture. We propose Graph Diffuser (GD) to address this challenge. GD learns to extract structural and positional relationships between distant nodes in the graph, which it then uses to direct the Transformer's attention and node representation. We demonstrate that existing GNNs and Graph Transformers struggle to capture long-range interactions and how Graph Diffuser does so while admitting intuitive visualizations. Experiments on eight benchmarks show Graph Diffuser to be a highly competitive model, outperforming the state-of-the-art in a diverse set of domains.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑