arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.05296cs.LGcs.AI
收藏

通过有根子树为图定制自注意力机制

Tailoring Self-Attention for Graph via Rooted Subtrees

Shanghai Jiaotong University(上海交通大学)

Siyuan Huang, Yunchong Song, Jiayue Zhou, Zhouhan Lin

AI总结:

针对现有图注意力机制长程信息捕获不足或局部细粒度信息缺失的问题,提出基于有根子树的多跳子树注意力STA及高效形式,构建STAGNN模型,在10个节点分类任务上性能优于现有图Transformer和主流GNN。

摘要:

注意力机制在图学习领域已取得显著进展,但仍存在明显局限:局部注意力受消息传递机制固有问题的制约,难以捕获长程信息;全局注意力无法体现层次化邻域结构,也难以捕获细粒度局部信息。本文提出一种新型多跳图注意力机制,命名为Subtree Attention(STA,子树注意力),以解决上述问题。STA将全注意力结构与有根子树无缝衔接,且经理论证明,STA在极端设置下可近似全局注意力。通过支持直接计算多跳邻居间的注意力权重,STA缓解了现有图注意力机制的固有缺陷。我们进一步通过引入核化softmax设计了STA的高效形式,使其具备线性时间复杂度。由此得到的图神经网络架构STAGNN,是一种利用跳感知注意力策略的简洁且高性能的基于STA的图神经网络。在10个节点分类数据集上的综合评估表明,基于STA的模型性能优于现有的图Transformer和主流GNN。代码可在https://github.com/LUMIA-Group/SubTree-Attention获取。

英文摘要:

Attention mechanisms have made significant strides in graph learning, yet they still exhibit notable limitations: local attention faces challenges in capturing long-range information due to the inherent problems of the message-passing scheme, while global attention cannot reflect the hierarchical neighborhood structure and fails to capture fine-grained local information. In this paper, we propose a novel multi-hop graph attention mechanism, named Subtree Attention (STA), to address the aforementioned issues. STA seamlessly bridges the fully-attentional structure and the rooted subtree, with theoretical proof that STA approximates the global attention under extreme settings. By allowing direct computation of attention weights among multi-hop neighbors, STA mitigates the inherent problems in existing graph attention mechanisms. Further we devise an efficient form for STA by employing kernelized softmax, which yields a linear time complexity. Our resulting GNN architecture, the STAGNN, presents a simple yet performant STA-based graph neural network leveraging a hop-aware attention strategy. Comprehensive evaluations on ten node classification datasets demonstrate that STA-based models outperform existing graph transformers and mainstream GNNs. The code is available at https://github.com/LUMIA-Group/SubTree-Attention.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑