Foundation models like ChatGPT and GPT-4 have revolutionized artificial intelligence, exhibiting remarkable abilities to generalize across a wide array of tasks and applications beyond their initial training objectives. However, graph learning has predominantly focused on single-graph models, tailored to specific tasks or datasets, lacking the ability to transfer learned knowledge to different domains. This limitation stems from the inherent complexity and diversity of graph structures, along with the different feature and label spaces specific to graph data. In this paper, we recognize text as an effective unifying medium and employ Text-Attributed Graphs (TAGs) to leverage this potential. We present our UniGraph framework, designed to learn a foundation model for TAGs, which is capable of generalizing to unseen graphs and tasks across diverse domains. Unlike single-graph models that use pre-computed node features of varying dimensions as input, our approach leverages textual features for unifying node representations, even for graphs such as molecular graphs that do not naturally have textual features. We propose a novel cascaded architecture of Language Models (LMs) and Graph Neural Networks (GNNs) as backbone networks. Additionally, we propose the first pre-training algorithm specifically designed for large-scale self-supervised learning on TAGs, based on Masked Graph Modeling. We introduce graph instruction tuning using Large Language Models (LLMs) to enable zero-shot prediction ability. Our comprehensive experiments across various graph learning tasks and domains demonstrate the model's effectiveness in self-supervised representation learning on unseen graphs, few-shot in-context transfer, and zero-shot transfer, even surpassing or matching the performance of GNNs that have undergone supervised training on target datasets.
Learning Noise-Resilient and Transferable Graph-Text Alignment via Dynamic Quality Assessment
School of New Media and Communication, Tianjin University（新媒体与传播学院，天津大学）；Baidu（百度）；Shanghai Jiao Tong University（上海交通大学）；School of Computer Science and Engineering, Beihang University（计算机科学与工程学院，北航）
Representation Learning of Structured Data for Medical Foundation Models
ASUS Intelligent Cloud Services (AICS)（华硕智能云服务（AICS））；Imperial College London（伦敦帝国学院）；University of Manchester（曼彻斯特大学）；Cheng Hsin General Hospital（程新总医院）；Crayon Software（克雷扬软件公司）
LLM as GNN: Graph Vocabulary Learning for Text-Attributed Graph Foundation Models
Rutgers University（罗格斯大学）；Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)（穆罕默德·本·扎耶德人工智能大学）；University of Science and Technology of China（中国科学技术大学）；Meta AI；North Carolina State University（北卡罗来纳州立大学）
School of Software, BNRist, THUIBCS, BLBCI, Tsinghua University（软件学院，BNRist，THUIBCS，BLBCI，清华大学）；Institute of Artificial Intelligence and Robotics, College of Artificial Intelligence, Xi’an Jiaotong University（人工智能与机器人研究院，人工智能学院，西安交通大学）；College of Mechatronics and Control Engineering, Shenzhen University（机械与控制工程学院，深圳大学）；Beijing Institute of Technology（北京理工大学）
GLIP-OOD: Zero-Shot Graph OOD Detection with Graph Foundation Model
University of Southern California（南加州大学）；University of Maryland, College Park（马里兰大学学院公园分校）；University of Illinois Chicago（伊利诺伊大学芝加哥分校）；Florida State University（佛罗里达州立大学）
