arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

QUARTET:四分支交叉注意力与随机游走轨迹增强关系图上的Transformer

QUARTET: Quad-branch cross-Attention and Random-walk Traces for Enhancing Transformers on Relational Graphs

Kyaw Hpone Myint, Nan Jiang, Xiang Li, Zhe Wu, Alexandre G. R. Day, Pranab Mohanty, Giri Iyengar

arXiv 2609.26855首次发表:更新:

发表机构

Capital One(第一资本)

机构由 AI 辅助整理,请以论文原文为准。

AI 中文总结

QUARTET通过因果随机游走采样和四分支交叉注意力,在局部与全局层面增强图Transformer,于RelBench v1分类任务中匹配或超越现有最先进模型。

AI 中文摘要

关系深度学习(RDL)将多表数据库建模为异构时序图,而图Transformer目前在RelBench等基准上取得了最先进的性能。然而,当前领先的模型RelGT存在两个关键局限:其随机局部采样器生成的子图连接松散,阻碍了消息传递;其全局注意力模块依赖单一的、基于种子特征的记忆,忽略了更广泛的宏观动态。为克服这些局限,我们引入了QUARTET,一种富有表现力的图Transformer架构,它在局部子图上应用完全自注意力,同时通过交叉注意力分支丰富全局上下文。具体而言,QUARTET采用基于最近截断个性化PageRank(PPR)的因果随机游走(CRW)采样器,以提取紧凑、对枢纽鲁棒且密集连接的局部子图,且无时间泄漏。同时,一个四分支交叉注意力模块从四个互补视角整合全局上下文:种子特征、种子拓扑、时序动态和协作动态。在RelBench v1分类任务中,QUARTET持续匹配或超越当前最先进的图Transformer基线(HGT和RelGT)。消融研究证实,CRW采样器显著丰富了局部邻域质量,而全局分支提供了必要的、任务特定的预测增益。

英文摘要

Relational Deep Learning (RDL) models multi-table databases as heterogeneous temporal graphs, and graph transformers currently achieve state-of-the-art performance on benchmarks like RelBench. However, the current leading model, RelGT, suffers from two key limitations: its random local sampler yields loosely connected subgraphs that hinder message passing, and its global attention module relies on a single, seed-feature-based memory that ignores broader macro-level dynamics. To overcome these limitations, we introduce QUARTET, an expressive graph transformer architecture that applies full self-attention on local subgraphs while enriching global context through cross-attention branches. Specifically, QUARTET employs a Causal Random Walk (CRW) sampler based on recency-truncated Personalized PageRank (PPR) to extract compact, hub-robust, and densely connected local subgraphs without temporal leakage. Concurrently, a quad-branch cross-attention module integrates global context from four complementary perspectives: seed feature, seed topology, temporal dynamics, and collaborative dynamics. Across the RelBench v1 classification tasks, QUARTET consistently matches or outperforms the current state-of-the-art graph transformer baselines (HGT and RelGT). Ablation studies confirm that the CRW sampler significantly enriches local neighborhood quality, while the global branches provide essential, task-specific predictive gains.

CommentsThis work has been accepted for main conference track at Learning on Graphs (LoG) 2026

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑