论图神经网络在持续学习中的局限性与经验回放
On the Limitation and Experience Replay for GNNs in Continual Learning
AI总结：
该研究首次探索节点级图持续学习中GNN的可学习性，发现结构偏移会严重影响其效果，进而提出结构演化感知经验回放方法SEA-ER，通过拓扑感知选样与结构对齐回放缓解遗忘与结构偏移影响，实验验证了理论与方法的有效性。
摘要：
持续学习旨在使模型能够从一系列任务中逐步获取信息，这一方法对众多随时间动态演化的现实系统至关重要。近期研究中，图神经网络（GNN）在节点级图持续学习（NGCL）领域的探索呈现激增态势，NGCL是一种实用但具挑战性的范式，涉及在节点相关任务上持续训练GNN。尽管针对NGCL中GNN的持续学习策略已有最新进展，但仍缺乏透彻的理论理解，尤其是关于其可学习性的研究。可学习性指是否存在一种学习算法，能从假设/权重空间中生成优良的候选模型，这对NGCL开发中的模型选择至关重要。本文首次对NGCL中GNN的可学习性展开理论探索，揭示出由于图数据的互联特性，结构偏移会对可学习性产生重大影响。具体而言，在发生显著结构变化的情况下，GNN可能不适用于NGCL，这凸显了管控结构偏移的必要性。为缓解结构偏移的影响，我们提出了一种名为结构演化感知经验回放（SEA-ER）的新型经验回放方法。SEA-ER具备一项创新的经验选择策略，该策略利用了GNN的拓扑感知能力，同时还配有独特的回放策略，采用结构对齐来有效对抗灾难性遗忘，并降低NGCL场景中结构偏移对GNN的影响。我们开展的大量实验验证了我们的理论见解以及SEA-ER的有效性。
英文摘要：
Continual learning seeks to empower models to progressively acquire information from a sequence of tasks. This approach is crucial for many real-world systems, which are dynamic and evolve over time. Recent research has witnessed a surge in the exploration of Graph Neural Networks (GNN) in Node-wise Graph Continual Learning (NGCL), a practical yet challenging paradigm involving the continual training of a GNN on node-related tasks. Despite recent advancements in continual learning strategies for GNNs in NGCL, a thorough theoretical understanding, especially regarding its learnability, is lacking. Learnability concerns the existence of a learning algorithm that can produce a good candidate model from the hypothesis/weight space, which is crucial for model selection in NGCL development. This paper introduces the first theoretical exploration of the learnability of GNN in NGCL, revealing that learnability is heavily influenced by structural shifts due to the interconnected nature of graph data. Specifically, GNNs may not be viable for NGCL under significant structural changes, emphasizing the need to manage structural shifts. To mitigate the impact of structural shifts, we propose a novel experience replay method termed Structure-Evolution-Aware Experience Replay (SEA-ER). SEA-ER features an innovative experience selection strategy that capitalizes on the topological awareness of GNNs, alongside a unique replay strategy that employs structural alignment to effectively counter catastrophic forgetting and diminish the impact of structural shifts on GNNs in NGCL. Our extensive experiments validate our theoretical insights and the effectiveness of SEA-ER.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
