arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2302.04181cs.LGcs.AIcs.NE
收藏

关注图Transformer

Attending to Graph Transformers

RWTH Aachen University(亚琛工业大学);Intel AI Lab(英特尔人工智能实验室);Isomorphic Labs(同构实验室)

Luis Müller, Mikhail Galkin, Christopher Morris, Ladislav Rampášek

AI总结:

本文系统梳理图Transformer架构分类、理论性质与编码方法,并通过实验评估其在图属性恢复、异配图处理和防止过度压缩方面的能力,为后续研究提供方向。

摘要:

近年来,用于图的Transformer架构作为图机器学习中已有技术(如消息传递图神经网络)的替代方案而出现。迄今为止,它们已在分子预测数据集等任务上展现出有前景的实证结果,这通常归因于它们能够规避图神经网络的缺陷,例如过度平滑和过度压缩。在此,我们推导出图Transformer架构的分类体系,为这一新兴领域带来一定秩序。我们概述其理论性质,综述结构编码和位置编码,并讨论针对重要图类的扩展,例如3D分子图。在实证方面,我们探究图Transformer在多大程度上能够恢复各种图属性,它们在异配图上的表现如何,以及它们在多大程度上能够防止过度压缩。此外,我们概述了开放性挑战和研究方向,以促进未来工作。我们的代码可在https://github.com/luis-mueller/probing-graph-transformers获取。

英文摘要:

Recently, transformer architectures for graphs emerged as an alternative to established techniques for machine learning with graphs, such as (message-passing) graph neural networks. So far, they have shown promising empirical results, e.g., on molecular prediction datasets, often attributed to their ability to circumvent graph neural networks' shortcomings, such as over-smoothing and over-squashing. Here, we derive a taxonomy of graph transformer architectures, bringing some order to this emerging field. We overview their theoretical properties, survey structural and positional encodings, and discuss extensions for important graph classes, e.g., 3D molecular graphs. Empirically, we probe how well graph transformers can recover various graph properties, how well they can deal with heterophilic graphs, and to what extent they prevent over-squashing. Further, we outline open challenges and research direction to stimulate future work. Our code is available at https://github.com/luis-mueller/probing-graph-transformers.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑