arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.09904cs.LG
收藏

通过两阶段交互增强学习小批量图Transformer

Learning a Mini-batch Graph Transformer via Two-stage Interaction Augmentation

Zhejiang University(浙江大学);Hangzhou City University(杭州城市大学);Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security(杭州高新区(滨江)区块链与数据安全研究院)

Wenda Li, Kaixuan Chen, Shunyu Liu, Tongya Zheng, Wenjie Huang, Mingli Song

AI总结:

针对现有小批量图Transformer存在的邻居信息损失/压缩及全局特征捕捉受限问题,提出含局部、全局两阶段交互增强的LGMformer模型,在10个基准节点分类任务上验证了有效性。

摘要:

小批量图Transformer(Mini-batch Graph Transformer,MGT)作为一种新兴的图学习模型,在半监督节点预测任务中展现出显著优势,兼具更高的计算效率与更强的模型鲁棒性。然而,现有处理局部信息的方法要么依赖采样,要么采用简单聚合,分别会导致关键邻居信息的丢失与压缩。此外,每个小批量内节点数量有限,制约了模型捕捉图全局特征的能力。本文提出LGMformer这一新型MGT模型,采用从局部到全局视角的两阶段增强交互策略,以解决上述瓶颈。局部交互增强(LIA)模块提出邻居-目标交互Transformer(NTIformer),用于深入理解邻居节点与目标节点之间的协同交互模式,生成具备局部有效性的token列表作为MGT的输入。与之相对,全局交互增强(GIA)模块采用交叉注意力机制,将全图原型信息融入目标节点表示,从而补充全局图信息以确保更全面的感知。借此,LGMformer在MGT范式下实现了节点表示的增强。在10个基准数据集上开展的节点分类相关实验结果,验证了所提方法的有效性。我们的代码可在https://github.com/l-wd/LGMformer获取。

英文摘要:

Mini-batch Graph Transformer (MGT), as an emerging graph learning model, has demonstrated significant advantages in semi-supervised node prediction tasks with improved computational efficiency and enhanced model robustness. However, existing methods for processing local information either rely on sampling or simple aggregation, which respectively result in the loss and squashing of critical neighbor information.Moreover, the limited number of nodes in each mini-batch restricts the model's capacity to capture the global characteristic of the graph. In this paper, we propose LGMformer, a novel MGT model that employs a two-stage augmented interaction strategy, transitioning from local to global perspectives, to address the aforementioned bottlenecks.The local interaction augmentation (LIA) presents a neighbor-target interaction Transformer (NTIformer) to acquire an insightful understanding of the co-interaction patterns between neighbors and the target node, resulting in a locally effective token list that serves as input for the MGT. In contrast, global interaction augmentation (GIA) adopts a cross-attention mechanism to incorporate entire graph prototypes into the target node epresentation, thereby compensating for the global graph information to ensure a more comprehensive perception. To this end, LGMformer achieves the enhancement of node representations under the MGT paradigm.Experimental results related to node classification on the ten benchmark datasets demonstrate the effectiveness of the proposed method. Our code is available at https://github.com/l-wd/LGMformer.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑