arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2510.04567cs.LGcs.AI
收藏

GILT:一种无需LLM、无需微调的图基础模型用于上下文学习

GILT: An LLM-Free, Tuning-Free Graph Foundational Model for In-Context Learning

Institute for Artificial Intelligence, Peking University(北京大学人工智能研究院);Wangxuan Institute of Computer Technology, Peking University(北京大学王宣计算机技术研究所)

Weishuo Ma, Yanbo Wang, Xiyuan Wang, Lei Zou, Muhan Zhang

AI总结:

提出GILT框架,通过基于令牌的上下文学习机制统一处理节点、边和图级别的分类任务,无需大语言模型或微调,实现高效泛化。

摘要:

图神经网络(GNN)是处理关系数据的强大工具,但通常难以泛化到未见过的图,从而催生了图基础模型(GFM)的发展。然而,当前的GFM面临图数据极端异质性的挑战,每个图可能具有独特的特征空间、标签集和拓扑结构。为此,出现了两种主要范式:第一种利用大语言模型(LLM),但本质上依赖于文本,因此难以处理海量图中的数值特征;第二种预训练基于结构的模型,但适应新任务通常需要昂贵的每图微调阶段,造成关键效率瓶颈。在这项工作中,我们超越了这些限制,引入了图上下文学习Transformer(GILT),这是一个基于无需LLM且无需微调架构的框架。GILT引入了一种新颖的基于令牌的框架用于图上的上下文学习(ICL),在统一框架中重新定义了跨节点、边和图级别的分类任务。该机制是处理异质性的关键,因为它设计用于操作通用数值特征。此外,它从上下文中动态理解类别语义的能力实现了无需微调的适应。全面实验表明,与基于LLM或基于微调的基线相比,GILT以显著更少的时间实现了更强的少样本性能,验证了我们方法的有效性。我们的代码可在https://github.com/yiming421/inductnode/获取。

英文摘要:

Graph Neural Networks (GNNs) are powerful tools for processing relational data but often struggle to generalize to unseen graphs, giving rise to the development of Graph Foundational Models (GFMs). However, current GFMs are challenged by the extreme heterogeneity of graph data, where each graph can possess a unique feature space, label set, and topology. To address this, two main paradigms have emerged. The first leverages Large Language Models (LLMs), but is fundamentally text-dependent, thus struggles to handle the numerical features in vast graphs. The second pre-trains a structure-based model, but the adaptation to new tasks typically requires a costly, per-graph tuning stage, creating a critical efficiency bottleneck. In this work, we move beyond these limitations and introduce \textbf{G}raph \textbf{I}n-context \textbf{L}earning \textbf{T}ransformer (GILT), a framework built on an LLM-free and tuning-free architecture. GILT introduces a novel token-based framework for in-context learning (ICL) on graphs, reframing classification tasks spanning node, edge and graph levels in a unified framework. This mechanism is the key to handling heterogeneity, as it is designed to operate on generic numerical features. Further, its ability to understand class semantics dynamically from the context enables tuning-free adaptation. Comprehensive experiments show that GILT achieves stronger few-shot performance with significantly less time than LLM-based or tuning-based baselines, validating the effectiveness of our approach. Our code is available at: https://github.com/yiming421/inductnode/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑