VecFormer: Towards Efficient and Generalizable Graph Transformer with Graph Token Attention
Zhejiang University(浙江大学);Westlake University(西湖大学);Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));Tongji University(同济大学);ShanghaiTech University(上海科技大学);Southern University of Science and Technology(南方科技大学);Jilin University(吉林大学)
Jingbo Zhou, Jun Xia, Siyuan Li, Yunfan Liu, Wenjun Wang, Yufei Huang, Changxi Chi, Mutian Hong, Zhuoli Ouyang, Shu Wang, Zhongqi Wang, Xingyu Wu, Chang Yu, Stan Z. Li
Graph Transformer has demonstrated impressive capabilities in the field of graph representation learning. However, existing approaches face two critical challenges: (1) most models suffer from exponentially increasing computational complexity, making it difficult to scale to large graphs; (2) attention mechanisms based on node-level operations limit the flexibility of the model and result in poor generalization performance in out-of-distribution (OOD) scenarios. To address these issues, we propose \textbf{VecFormer} (the \textbf{Vec}tor Quantized Graph Trans\textbf{former}), an efficient and highly generalizable model for node classification, particularly under OOD settings. VecFormer adopts a two-stage training paradigm. In the first stage, two codebooks are used to reconstruct the node features and the graph structure, aiming to learn the rich semantic \texttt{Graph Codes}. In the second stage, attention mechanisms are performed at the \texttt{Graph Token} level based on the transformed cross codebook, reducing computational complexity while enhancing the model's generalization capability. Extensive experiments on datasets of various sizes demonstrate that VecFormer outperforms the existing Graph Transformer in both performance and speed.
Token-Level Prompt Mixture with Parameter-Free Routing for Federated Domain Generalization
School of Computing and Artificial Intelligence, Shandong University of Finance and Economics(计算机与人工智能学院,山东财经大学);School of Computer Science and Technology, Shandong Jianzhu University(计算机科学与技术学院,山东建筑大学);College of Electronics and Information Engineering, Tongji University(电子信息工程学院,同济大学);School of Information Science and Technology, University of Science and Technology of China(信息科学与技术学院,中国科学技术大学)
LMAE4Eth: Generalizable and Robust Ethereum Fraud Detection by Exploring Transaction Semantics and Masked Graph Embedding
Yantai Research Institute, Harbin Engineering University(哈尔滨工程大学烟台研究院);Hangzhou Research Institute, Xidian University(西安电子科技大学杭州研究院);College of Computer Science, Zhejiang University(浙江大学计算机学院)
VolTA-3D: Self-Supervised Learning for Brain MRI using 3D Volumetric Token Alignment
Institute of Health Informatics(健康信息学研究所);Sheikh Zayed Institute for Pediatric Surgical Innovation(谢赫扎耶德儿童外科创新研究所);School of Medicine and Health Sciences(医学与健康科学学院)
Functional Attention: From Pairwise Affinities to Functional Correspondences
Technical University of Munich, Germany(慕尼黑技术大学,德国);Munich Center for Machine Learning (MCML), Germany(慕尼黑机器学习中心(MCML),德国);PIXL, Department of Computer Science, University of Oxford, United Kingdom(牛津大学计算机科学系PIXL,英国);ECE, University of Texas at Austin, USA(德克萨斯大学奥斯汀分校电子与计算机工程系,美国)