arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.16699cs.LGcs.AI
收藏

Graph Transformers 梦想电力流

Graph Transformers Dream of Electric Flow

Duke University(杜克大学);Technical University of Munich(慕尼黑工业大学)

Xiang Cheng, Lawrence Carin, Suvrit Sra

AI总结:

该研究从理论和实验证明线性 Transformer 可通过图的关联矩阵实现电力流和特征向量分解算法,并发现其在分子回归任务中能学习比默认更优的位置编码。

摘要:

我们从理论和实验两方面证明,将线性 Transformer 应用于图数据时,能够实现解决电力流和特征向量分解等典型问题的算法。Transformer 仅通过图的关联矩阵获取输入图的信息。我们给出了实现每种算法的显式权重配置,并将所构建 Transformer 的误差限制在底层算法的误差范围内。我们在合成数据上的实验证实了理论发现。此外,在真实世界的分子回归任务中,我们观察到线性 Transformer 能够学习到比基于 Laplacian 特征向量的默认位置编码更有效的位置编码。我们的工作是阐明 Transformer 处理图数据内部机制的开端。代码可在 https://github.com/chengxiang/LinearGraphTransformer 获取。

英文摘要:

We show theoretically and empirically that the linear Transformer, when applied to graph data, can implement algorithms that solve canonical problems such as electric flow and eigenvector decomposition. The Transformer has access to information on the input graph only via the graph's incidence matrix. We present explicit weight configurations for implementing each algorithm, and we bound the constructed Transformers' errors by the errors of the underlying algorithms. Our theoretical findings are corroborated by experiments on synthetic data. Additionally, on a real-world molecular regression task, we observe that the linear Transformer is capable of learning a more effective positional encoding than the default one based on Laplacian eigenvectors. Our work is an initial step towards elucidating the inner-workings of the Transformer for graph data. Code is available at https://github.com/chengxiang/LinearGraphTransformer

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑