基于预训练模型的持续学习：综述
Continual Learning with Pre-Trained Models: A Survey
AI总结：
本文综述了基于预训练模型的持续学习最新进展，将现有方法分为三类并比较其优劣，同时通过实证研究指出了方法对比中的公平性问题。
摘要：
如今，现实应用常面临流数据，要求学习系统在数据演进时吸收新知识。持续学习（CL）旨在实现此目标，同时克服学习新知识时对旧知识的灾难性遗忘。典型的CL方法从零构建模型以随新数据增长。然而，预训练模型（PTM）时代的到来激发了巨大的研究兴趣，特别是利用PTM强大的表征能力。本文对基于PTM的CL最新进展进行了全面综述。我们将现有方法分为三个不同组别，对它们的相似性、差异及各自的优缺点进行了比较分析。此外，我们提供了一项对比各种最先进方法的实证研究，以突出比较中关于公平性的担忧。用于复现这些评估的源代码可在：https://github.com/sun-hailong/LAMDA-PILOT获取。
英文摘要：
Nowadays, real-world applications often face streaming data, which requires the learning system to absorb new knowledge as data evolves. Continual Learning (CL) aims to achieve this goal and meanwhile overcome the catastrophic forgetting of former knowledge when learning new ones. Typical CL methods build the model from scratch to grow with incoming data. However, the advent of the pre-trained model (PTM) era has sparked immense research interest, particularly in leveraging PTMs' robust representational capabilities. This paper presents a comprehensive survey of the latest advancements in PTM-based CL. We categorize existing methodologies into three distinct groups, providing a comparative analysis of their similarities, differences, and respective advantages and disadvantages. Additionally, we offer an empirical study contrasting various state-of-the-art methods to highlight concerns regarding fairness in comparisons. The source code to reproduce these evaluations is available at: https://github.com/sun-hailong/LAMDA-PILOT
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
