arXiv 2505.186972025-09-26cs.LGcs.AI

大型语言模型能否缓解图持续学习中的灾难性遗忘？一项系统性研究

Can LLMs Alleviate Catastrophic Forgetting in Graph Continual Learning? A Systematic Study

The Chinese University of Hong Kong（香港中文大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；University of Science and Technology of China（中国科学技术大学）；University of Pittsburgh（匹兹堡大学）；Tongji University（同济大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）

Ziyang Cheng, Zhixun Li, Yuhan Li, Yixin Song, Kangyi Zhao, Dawei Cheng, Jia Li, Hong Cheng, Jeffrey Xu Yu

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本研究系统评估LLMs在GCL中的表现，指出任务ID泄露问题，提出SimGCL方法，在无重放约束下超越SOTA约20%，并发布基准LLM4GCL。

摘要：

如今，现实世界中的数据（包括图结构数据）通常以流式方式到达，这意味着学习系统需要在不遗忘先前所学信息的情况下持续获取新知识。尽管已有大量工作试图解决图机器学习中的灾难性遗忘问题，但它们都基于使用流式数据从头训练。随着预训练模型的兴起，越来越多的研究利用其强大的泛化能力进行持续学习。因此，在本工作中，我们试图回答大型语言模型（LLMs）能否缓解图持续学习（GCL）中的灾难性遗忘。我们首先指出当前GCL的实验设置存在显著缺陷，因为评估阶段可能导致任务ID泄露。然后，我们在更现实的场景中评估LLMs的性能，发现即使微小的修改也能带来出色的结果。最后，基于大量实验，我们提出了一种简单而有效的方法，即简单图持续学习（SimGCL），在无重放约束下，该方法比之前最先进的基于GNN的基线高出约20%。为促进可复现性，我们开发了一个易于使用的基准LLM4GCL，用于训练和评估现有的GCL方法。代码可在https://github.com/ZhixunLEE/LLM4GCL获取。

英文摘要：

Nowadays, real-world data, including graph-structure data, often arrives in a streaming manner, which means that learning systems need to continuously acquire new knowledge without forgetting previously learned information. Although substantial existing works attempt to address catastrophic forgetting in graph machine learning, they are all based on training from scratch with streaming data. With the rise of pretrained models, an increasing number of studies have leveraged their strong generalization ability for continual learning. Therefore, in this work, we attempt to answer whether large language models (LLMs) can mitigate catastrophic forgetting in Graph Continual Learning (GCL). We first point out that current experimental setups for GCL have significant flaws, as the evaluation stage may lead to task ID leakage. Then, we evaluate the performance of LLMs in more realistic scenarios and find that even minor modifications can lead to outstanding results. Finally, based on extensive experiments, we propose a simple-yet-effective method, Simple Graph Continual Learning (SimGCL), that surpasses the previous state-of-the-art GNN-based baseline by around 20% under the rehearsal-free constraint. To facilitate reproducibility, we have developed an easy-to-use benchmark LLM4GCL for training and evaluating existing GCL methods. The code is available at: https://github.com/ZhixunLEE/LLM4GCL.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。