立场:图基础模型已经到来
Position: Graph Foundation Models are Already Here
AI总结:
本文提出图基础模型(GFMs)开发的新视角,倡导构建“图词汇表”以编码图上的不变性,从而有效利用多样化图数据实现正向迁移,推动符合神经缩放定律的GFM设计。
摘要:
图基础模型(GFMs)正成为图领域中重要的研究课题,旨在开发基于广泛且多样化数据训练的图模型,以增强其在各种任务和领域的适用性。与通常在特定数据集上针对特定任务从头开始训练的传统图神经网络(GNNs)相比,开发GFMs面临着独特的挑战。构建GFMs的主要挑战在于有效利用海量且多样化的图数据以实现正向迁移。受CV和NLP领域中现有基础模型的启发,我们提出了一种新的GFM开发视角,倡导构建“图词汇表”,其中图底层的基本可迁移单元编码了图上的不变性。我们从网络分析、表达能力和稳定性等关键方面为图词汇表的构建奠定了基础。这种词汇表视角有望推动未来的GFM设计符合神经缩放定律。所有相关的GFM设计资源均可在此处找到。
英文摘要:
Graph Foundation Models (GFMs) are emerging as a significant research topic in the graph domain, aiming to develop graph models trained on extensive and diverse data to enhance their applicability across various tasks and domains. Developing GFMs presents unique challenges over traditional Graph Neural Networks (GNNs), which are typically trained from scratch for specific tasks on particular datasets. The primary challenge in constructing GFMs lies in effectively leveraging vast and diverse graph data to achieve positive transfer. Drawing inspiration from existing foundation models in the CV and NLP domains, we propose a novel perspective for the GFM development by advocating for a ``graph vocabulary'', in which the basic transferable units underlying graphs encode the invariance on graphs. We ground the graph vocabulary construction from essential aspects including network analysis, expressiveness, and stability. Such a vocabulary perspective can potentially advance the future GFM design in line with the neural scaling laws. All relevant resources with GFM design can be found here.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。