arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.07654cs.LG
收藏

图三元注意力网络:一种解耦视角

Graph Triple Attention Network: A Decoupled Perspective

Huazhong University of Science and Technology(华中科技大学);Zhejiang University(浙江大学);Rutgers University(罗格斯大学);Ant Group(蚂蚁集团)

Xiaotang Wang, Yun Zhu, Haizhou Shi, Yongchao Liu, Chuntao Hong

AI总结:

本文提出解耦视角下的图三元注意力网络DeGTA,通过分离位置、结构和属性注意力并自适应整合局部与全局信息,在节点和图分类任务上取得最先进性能。

摘要:

图Transformer(GTs)近年来通过有效捕获长程依赖和图归纳偏置,在图领域取得了显著成功。然而,这些方法面临两个主要挑战:(1)多视图混乱,源于将多视图信息(位置、结构、属性)耦合在一起,从而阻碍了灵活使用和传播过程的可解释性。(2)局部-全局混乱,源于将局部消息传递与全局注意力耦合,导致过拟合和过度全局化问题。为应对这些挑战,我们提出了一种高层次的GTs解耦视角,将其分解为三个组件和两个交互层级:位置注意力、结构注意力和属性注意力,以及局部和全局交互。基于这一解耦视角,我们设计了一个名为DeGTA的解耦图三元注意力网络,它分别计算多视图注意力,并自适应地整合多视图局部和全局信息。该方法具有三个关键优势:增强的可解释性、灵活的设计以及局部与全局信息的自适应整合。通过大量实验,DeGTA在包括节点分类和图分类在内的各种数据集和任务上取得了最先进的性能。全面的消融研究表明,解耦对于提升性能和增强可解释性至关重要。我们的代码可在以下地址获取:https://github.com/wangxiaotang0906/DeGTA

英文摘要:

Graph Transformers (GTs) have recently achieved significant success in the graph domain by effectively capturing both long-range dependencies and graph inductive biases. However, these methods face two primary challenges: (1) multi-view chaos, which results from coupling multi-view information (positional, structural, attribute), thereby impeding flexible usage and the interpretability of the propagation process. (2) local-global chaos, which arises from coupling local message passing with global attention, leading to issues of overfitting and over-globalizing. To address these challenges, we propose a high-level decoupled perspective of GTs, breaking them down into three components and two interaction levels: positional attention, structural attention, and attribute attention, alongside local and global interaction. Based on this decoupled perspective, we design a decoupled graph triple attention network named DeGTA, which separately computes multi-view attentions and adaptively integrates multi-view local and global information. This approach offers three key advantages: enhanced interpretability, flexible design, and adaptive integration of local and global information. Through extensive experiments, DeGTA achieves state-of-the-art performance across various datasets and tasks, including node classification and graph classification. Comprehensive ablation studies demonstrate that decoupling is essential for improving performance and enhancing interpretability. Our code is available at: https://github.com/wangxiaotang0906/DeGTA

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑