arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2505.02027cs.LGcs.AIcs.SI
收藏

GraphPrompter:面向图上下文学习的多阶段自适应提示优化

GraphPrompter: Multi-stage Adaptive Prompt Optimization for Graph In-Context Learning

State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China(认知智能国家重点实验室,中国科学技术大学)

Rui Lv, Zaixi Zhang, Kai Zhang, Qi Liu, Weibo Gao, Jiawei Liu, Jiaxia Yan, Linan Yue, Fangzhou Yao

AI总结:

GraphPrompter 通过多阶段自适应优化图提示的生成、选择与增强,解决图上下文学习中提示噪声和类别分布差异问题,平均性能超越现有最优基线 8% 以上。

摘要:

图上下文学习(Graph In-Context Learning)能够在无需更新任何参数的情况下,将预训练图模型适配到新颖且多样的下游图,因而受到社区的广泛关注。图上下文学习的关键在于基于所选择的提示示例对下游图进行条件化处理。现有方法随机选择子图或边作为提示,导致图提示存在噪声,模型性能较差。此外,由于预训练图与测试图之间存在差距,当测试图中的类别数量远大于训练图时,上下文学习能力也会显著下降。为解决上述挑战,我们提出了一种多阶段自适应提示优化方法 GraphPrompter,该方法优化图提示的生成、选择和使用全过程,以获得更好的上下文学习能力。首先,Prompt Generator 引入重建层,以突出最具信息量的边并减少无关噪声,用于图提示构建。其次,在选择阶段,Prompt Selector 采用 $k$-最近邻算法和预训练选择层,动态选择合适样本,并最小化无关提示的影响。最后,我们利用带有缓存替换策略的 Prompt Augmenter 来增强预训练模型在新数据集上的泛化能力。大量实验表明,GraphPrompter 有效增强了图模型的上下文学习能力。在所有设置的平均结果上,我们的方法比最先进的基线高出 8% 以上。我们的代码已发布在 https://github.com/karin0018/GraphPrompter。

英文摘要:

Graph In-Context Learning, with the ability to adapt pre-trained graph models to novel and diverse downstream graphs without updating any parameters, has gained much attention in the community. The key to graph in-context learning is to perform downstream graphs conditioned on chosen prompt examples. Existing methods randomly select subgraphs or edges as prompts, leading to noisy graph prompts and inferior model performance. Additionally, due to the gap between pre-training and testing graphs, when the number of classes in the testing graphs is much greater than that in the training, the in-context learning ability will also significantly deteriorate. To tackle the aforementioned challenges, we develop a multi-stage adaptive prompt optimization method GraphPrompter, which optimizes the entire process of generating, selecting, and using graph prompts for better in-context learning capabilities. Firstly, Prompt Generator introduces a reconstruction layer to highlight the most informative edges and reduce irrelevant noise for graph prompt construction. Furthermore, in the selection stage, Prompt Selector employs the $k$-nearest neighbors algorithm and pre-trained selection layers to dynamically choose appropriate samples and minimize the influence of irrelevant prompts. Finally, we leverage a Prompt Augmenter with a cache replacement strategy to enhance the generalization capability of the pre-trained model on new datasets. Extensive experiments show that GraphPrompter effectively enhances the in-context learning ability of graph models. On average across all the settings, our approach surpasses the state-of-the-art baselines by over 8%. Our code is released at https://github.com/karin0018/GraphPrompter.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑