Unified Multi-Domain Graph Pre-training for Homogeneous and Heterogeneous Graphs via Domain-Specific Expert Encoding
School of Computer Science;Technology Tianjin University Tianjin China;North China University of Science;School of Economics;Management Beijing University of Posts;Departments of Health Technology \& Informatics;Computing The Hong Kong Polytechnic University Kowloon Hong Kong;Tianjin University;Beijing University of Posts;The Hong Kong Polytechnic University
Graph pre-training has achieved remarkable success in recent years, delivering transferable representations for downstream adaptation. However, most existing methods are designed for either homogeneous or heterogeneous graphs, thereby hindering unified graph modeling across diverse graph types. This separation contradicts real-world applications, where mixed homogeneous and heterogeneous graphs are ubiquitous, and distribution shifts between upstream pre-training and downstream deployment are common. In this paper, we empirically demonstrate that a balanced mixture of homogeneous and heterogeneous graph pre-training benefits downstream tasks and propose a unified multi-domain \textbf{G}raph \textbf{P}re-training method across \textbf{H}omogeneous and \textbf{H}eterogeneous graphs ($\mathbf{GPH^{2}}$). To address the lack of a unified encoder for homogeneous and heterogeneous graphs, we propose a Unified Multi-View Graph Construction that simultaneously encodes both without explicit graph-type-specific designs. To cope with the increased cross-domain distribution discrepancies arising from mixed graphs, we introduce domain-specific expert encoding. Each expert is independently pre-trained on a single graph to capture domain-specific knowledge, thereby shielding the pre-training encoder from the adverse effects of cross-domain discrepancies. For downstream tasks, we further design a Task-oriented Expert Fusion Strategy that adaptively integrates multiple experts based on their discriminative strengths. Extensive experiments on mixed graphs demonstrate that $\text{GPH}^{2}$ enables stable transfer across graph types and domains, significantly outperforming existing graph pre-training methods.
Adaptive Graph Mixture of Residual Experts: Unsupervised Learning on Diverse Graphs with Heterogeneous Specialization
School of New Media and Communication, Tianjin University(新媒体与传播学院,天津大学);Baidu(百度);School of Computer Science and Engineering, Beihang University(计算机科学与工程学院,北航大学)
Scalable Heterogeneous Graph Learning via Heterogeneous-aware Orthogonal Prototype Experts
National Engineering Research Center for Big Data Technology;System Lab, Cluster;Grid Computing Lab, School of Computer Science;Technology, Huazhong University of Science;DIRO, Université de Montréal \& Mila \& Canada CIFAR AI Chair, Canada
Effective MoE-based LLM Compression by Exploiting Heterogeneous Inter-Group Experts Routing Frequency and Information Density
Department of Computer Science, Stevens Institute of Technology, Hoboken, NJ, USA(计算机科学系,史蒂文斯理工学院,新泽西州霍博肯);Northeastern University, Boston, MA, USA(东北大学,波士顿,马萨诸塞州)
UB-SMoE: Universally Balanced Sparse Mixture-of-Experts for Resource-adaptive Federated Fine-tuning of Foundation Models
School of Computer Science and Statistics, Trinity College Dublin, Ireland(特里尼蒂学院都柏林分校计算机科学与统计学学院);School of Computer Science, University College Dublin, Ireland(都柏林大学学院计算机科学学院)
Personalized Federated Learning on Heterogeneous and Long-Tailed Data via Expert Collaborative Learning
Xiamen University(厦门大学);University College London(伦敦大学学院);A*STAR Institute of High Performance Computing(新加坡高性能计算研究院);Guangdong University of Technology(广东工业大学);Guangdong Laboratory of Artificial Intelligence and Digital Economy(广东省人工智能与数字经济实验室)
CHoE: Cross-Domain Heterogeneous Graph Prompt Learning via Structure-Conditioned Experts
School of Computer Science and Technology, Tianjin University(天津大学计算机科学与技术学院);Department of Health Technology and Informatics, and Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University(香港理工大学健康科技与信息学系、数据科学与人工智能系)
FedRE: A Representation Entanglement Framework for Model-Heterogeneous Federated Learning
Teleinfo, CAICT(中国信息通信研究院);Nanyang Technological University(南洋理工大学);Tsinghua University(清华大学);Nanjing University of Posts and Telecommunications(南京邮电大学);University of Texas at Dallas(德克萨斯大学达拉斯分校);University of British Columbia(不列颠哥伦比亚大学);China University of Mining and Technology-Beijing(中国矿业大学(北京));China University of Mining and Technology(中国矿业大学);Beijing Jiaotong University(北京交通大学)
N-BEATS-MOE: N-BEATS with a Mixture-of-Experts Layer for Heterogeneous Time Series Forecasting
Faculdade de Engenharia da Universidade do Porto, Porto, Portugal(葡萄牙波尔图大学工程学院);Laboratory for Artificial Intelligence and Computer Science (LIACC)(人工智能与计算机科学实验室)
CausalMoE: A Billion-Scale Multimodal Foundation Model for Granger Causal Discovery with Pattern-Routed Heterogeneous Experts
State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University(北京大学智能科学与技术学院通用人工智能国家重点实验室);National Institute of Health Data Science, and Institute for Artificial Intelligence, Peking University(北京大学健康医疗大数据国家研究院、人工智能研究院)
Adaptive Expert Models for Personalization in Federated Learning
Ericsson AB(爱立信公司);KTH Royal Institute of Technology(KTH皇家理工学院);RISE Research Institutes of Sweden(瑞典RISE研究院);AI Sweden(瑞典人工智能研究院);Chalmers University of Technology(查尔姆斯理工大学)
Exploiting Heterogeneous Graph Neural Networks with Latent Worker/Task Correlation Information for Label Aggregation in Crowdsourcing
Zhejiang University(浙江大学);IBM T. J. Watson Research Center(IBM托马斯·J·沃森研究中心);JD.COM Silicon Valley Research Center(京东硅谷研发中心);Squirrel AI Learning(松鼠AI智适应教育)
Linking heterogeneous microstructure informatics with expert characterization knowledge through customized and hybrid vision-language representations for industrial qualification
McGill University(麦吉尔大学);National Research Council Canada(加拿大国家研究委员会);Apollo-Clad Laser Cladding, a division of Apollo Machine and Welding Ltd.(Apollo-Clad激光熔覆,Apollo机械与焊接有限公司旗下部门);Fraunhofer Institute for Laser Technology ILT(弗劳恩霍夫激光技术研究所)
Mixture of Weight-shared Heterogeneous Group Attention Experts for Dynamic Token-wise KV Optimization
University of Chinese Academy of Sciences(中国科学院大学);Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);University of Macau(澳门大学);Shenzhen University of Advanced Technology(深圳先进技术大学)
Mosaic: Data-Free Knowledge Distillation via Mixture-of-Experts for Heterogeneous Distributed Environments
School of Computer Science and Technology, Tongji University(同济大学计算机科学与技术学院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);The City University of New York(纽约城市大学);Shenzhen University of Advanced Technology(深圳先进技术大学)
School of Computer Science, Wuhan University(武汉大学计算机学院);School of Computing, Macquarie University(麦考瑞大学计算学院);School of Information and Communication Technology, Griffith University(格里菲斯大学信息与通信技术学院);Faculty of Engineering and Information Technology, University of Technology Sydney(悉尼科技大学工程与信息技术学院)