arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.08310cs.LG
收藏

GraphFM:面向图基础模型的综合基准

GraphFM: A Comprehensive Benchmark for Graph Foundation Model

New York University Shanghai(上海纽约大学);National University of Singapore(新加坡国立大学);Rice University(莱斯大学)

Yuhao Xu, Xinqi Liu, Keyu Duan, Yi Fang, Yu-Neng Chuang, Daochen Zha, Qiaoyu Tan

AI总结:

本文提出GraphFM基准,系统评估自监督GNN图基础模型的泛化性、可扩展性、效率及预训练停止策略,并开源代码。

摘要:

基础模型(FMs)是开发人工智能系统的一大通用类别,为在一系列下游任务上实现泛化提供了广泛潜力。尽管作为FMs基石的自监督学习已得到广泛研究,但依赖图自监督学习的图基础模型仍存在若干突出问题:1)同质化。其在下游任务上的泛化能力程度仍不清楚。2)可扩展性。这些模型能否有效扩展到大型数据集尚属未知。3)效率。这些模型的训练时间和内存使用需要评估。4)训练停止准则。需要确定跨多个任务进行预训练时的最优停止策略,以最大化下游任务性能。为解决这些问题,我们构建了一个严格的基准,全面分析和研究自监督图神经网络(GNN)模型的泛化性与可扩展性。在泛化性方面,我们实现并比较了多种自监督GNN模型的性能,这些模型经过训练以生成节点表示,并在节点分类、链路预测和节点聚类等任务上进行评估。在可扩展性方面,我们比较了各种模型使用全批次和小批次策略训练后的性能。此外,我们通过实验测试这些模型的GPU内存使用量和吞吐量,评估了其训练效率。通过这些实验,我们旨在提供见解以推动未来研究。该基准的代码已公开发布于https://github.com/NYUSHCS/GraphFM。

英文摘要:

Foundation Models (FMs) serve as a general class for the development of artificial intelligence systems, offering broad potential for generalization across a spectrum of downstream tasks. Despite extensive research into self-supervised learning as the cornerstone of FMs, several outstanding issues persist in Graph Foundation Models that rely on graph self-supervised learning, namely: 1) Homogenization. The extent of generalization capability on downstream tasks remains unclear. 2) Scalability. It is unknown how effectively these models can scale to large datasets. 3) Efficiency. The training time and memory usage of these models require evaluation. 4) Training Stop Criteria. Determining the optimal stopping strategy for pre-training across multiple tasks to maximize performance on downstream tasks. To address these questions, we have constructed a rigorous benchmark that thoroughly analyzes and studies the generalization and scalability of self-supervised Graph Neural Network (GNN) models. Regarding generalization, we have implemented and compared the performance of various self-supervised GNN models, trained to generate node representations, across tasks such as node classification, link prediction, and node clustering. For scalability, we have compared the performance of various models after training using full-batch and mini-batch strategies. Additionally, we have assessed the training efficiency of these models by conducting experiments to test their GPU memory usage and throughput. Through these experiments, we aim to provide insights to motivate future research. The code for this benchmark is publicly available at https://github.com/NYUSHCS/GraphFM.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑