正则化图持续学习的无偏在线曲率近似
Unbiased Online Curvature Approximation for Regularized Graph Continual Learning
AI总结：
针对图持续学习中的灾难性遗忘问题，提出基于Fisher信息矩阵的无偏在线曲率近似正则化方法，无需显式存储FIM即可在线估计正则化项，在稳定性和可塑性间实现更优权衡。
摘要：
图持续学习（GCL）旨在从连续的基于图的任务序列中进行学习。正则化方法对于防止GCL中的灾难性遗忘至关重要，特别是在具有挑战性的无回放、类增量设置中，其中每个任务由一组独特的类组成。在这项工作中，我们首先基于由Fisher信息矩阵（FIM）诱导的弯曲参数空间，为GCL建立了一个通用正则化框架。我们证明了占主导地位的弹性权重巩固（EWC）及其变体是该框架内的一个特例，它们使用了基于先前任务参数的经验FIM的对角近似。为了克服它们的局限性，我们提出了一种基于模型当前学习状态的完整FIM的无偏在线曲率近似。我们的方法以在线方式直接估计正则化项，而无需显式评估和存储FIM本身。这使得模型在学习新任务时能够更好地捕捉损失景观，同时保留从先前任务中学到的知识。在三个图数据集上的大量实验表明，我们的方法显著优于现有的基于正则化的方法，在稳定性（保留旧知识）和可塑性（获取新知识）之间实现了更优的权衡。
英文摘要：
Graph continual learning (GCL) aims to learn from a continuous sequence of graph-based tasks. Regularization methods are vital for preventing catastrophic forgetting in GCL, particularly in the challenging replay-free, class-incremental setting, where each task consists of a set of unique classes. In this work, we first establish a general regularization framework for GCL based on the curved parameter space induced by the Fisher information matrix (FIM). We show that the dominant Elastic Weight Consolidation (EWC) and its variants are a special case within this framework, using a diagonal approximation of the empirical FIM based on parameters from previous tasks. To overcome their limitations, we propose a new unbiased online curvature approximation of the full FIM based on the model's current learning state. Our method directly estimates the regularization term in an online manner without explicitly evaluating and storing the FIM itself. This enables the model to better capture the loss landscape during learning new tasks while retaining the knowledge learned from previous tasks. Extensive experiments on three graph datasets demonstrate that our method significantly outperforms existing regularization-based methods, achieving a superior trade-off between stability (retaining old knowledge) and plasticity (acquiring new knowledge).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
