提示驱动的持续图学习
Prompt-Driven Continual Graph Learning
AI总结:
针对持续图学习中记忆回放策略的可扩展性与隐私问题,提出PROMPTCGL框架,通过分层提示(特征+拓扑)和个性化提示生成器,固定模型并为新任务学提示,实现低内存消耗下的更优性能。
摘要:
持续图学习(CGL)旨在让模型在不断演化的图数据上适应新任务,同时不遗忘已有知识,目前正受到广泛研究关注。主流解决方案采用基于记忆回放的思路,即缓存早期任务的代表性数据以重新训练图模型。然而,该策略在处理持续演化的图时面临可扩展性问题,且引发数据隐私担忧。受基于提示的学习范式最新进展启发,本文提出一种新颖的提示驱动持续图学习(PROMPTCGL)框架,为每个新任务学习单独的提示,并保持底层图神经网络模型固定。通过这种方式,PROMPTCGL自然避免了对先前任务知识的灾难性遗忘。更具体地说,我们提出分层提示,从特征和拓扑层面指导模型,以充分应对动态持续学习中任务图的变异性。此外,我们开发了个性化提示生成器,为每个图节点生成定制提示,同时最小化所需提示数量,实现了无论图规模如何都保持恒定的内存消耗。在四个基准数据集上的大量实验表明,PROMPTCGL相比现有CGL方法取得了更优性能,同时显著降低了内存消耗。我们的代码可在https://github.com/QiWang98/PromptCGL获取。
英文摘要:
Continual Graph Learning (CGL), which aims to accommodate new tasks over evolving graph data without forgetting prior knowledge, is garnering significant research interest. Mainstream solutions adopt the memory replay-based idea, ie, caching representative data from earlier tasks for retraining the graph model. However, this strategy struggles with scalability issues for constantly evolving graphs and raises concerns regarding data privacy. Inspired by recent advancements in the prompt-based learning paradigm, this paper introduces a novel prompt-driven continual graph learning (PROMPTCGL) framework, which learns a separate prompt for each incoming task and maintains the underlying graph neural network model fixed. In this way, PROMPTCGL naturally avoids catastrophic forgetting of knowledge from previous tasks. More specifically, we propose hierarchical prompting to instruct the model from both feature- and topology-level to fully address the variability of task graphs in dynamic continual learning. Additionally, we develop a personalized prompt generator to generate tailored prompts for each graph node while minimizing the number of prompts needed, leading to constant memory consumption regardless of the graph scale. Extensive experiments on four benchmarks show that PROMPTCGL achieves superior performance against existing CGL approaches while significantly reducing memory consumption. Our code is available at https://github.com/QiWang98/PromptCGL.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。