在层间细读：利用多层表示实现基于预训练模型的无排练持续学习
Read Between the Layers: Leveraging Multi-Layer Representations for Rehearsal-Free Continual Learning with Pre-Trained Models
AI总结：
本文提出LayUP方法，通过利用预训练模型多个中间层的二阶特征统计量进行基于原型的持续学习，无需排练即可在多数类增量、域增量和在线持续学习基准上超越现有方法，并显著降低内存和计算开销。
摘要：
我们研究持续学习（CL）问题，即模型必须从非平稳分布中学习一系列任务，同时在遇到新经验时保留先前知识。随着基础模型的发展，持续学习研究已从最初的从零学习范式转向利用大规模预训练的通用特征。然而，现有的基于预训练模型的持续学习方法主要侧重于从最终表示层中分离类别特定特征，而忽视了中间表示在捕获低层和中层特征方面的潜力，这些特征对域偏移更为不变。在本工作中，我们提出LayUP，这是一种新的基于原型的持续学习方法，利用预训练网络多个中间层的二阶特征统计量。我们的方法概念简单，不需要访问先前数据，并且可与任何基础模型开箱即用。LayUP在七个类增量学习基准中的四个、全部三个域增量学习基准以及七个在线持续学习基准中的六个上超越了最先进水平，同时与现有基线相比显著降低了内存和计算需求。我们的结果表明，在持续学习中充分挖掘预训练模型的表示能力远不止于其最终嵌入。
英文摘要：
We address the Continual Learning (CL) problem, wherein a model must learn a sequence of tasks from non-stationary distributions while preserving prior knowledge upon encountering new experiences. With the advancement of foundation models, CL research has pivoted from the initial learning-from-scratch paradigm towards utilizing generic features from large-scale pre-training. However, existing approaches to CL with pre-trained models primarily focus on separating class-specific features from the final representation layer and neglect the potential of intermediate representations to capture low- and mid-level features, which are more invariant to domain shifts. In this work, we propose LayUP, a new prototype-based approach to CL that leverages second-order feature statistics from multiple intermediate layers of a pre-trained network. Our method is conceptually simple, does not require access to prior data, and works out of the box with any foundation model. LayUP surpasses the state of the art in four of the seven class-incremental learning benchmarks, all three domain-incremental learning benchmarks and in six of the seven online continual learning benchmarks, while significantly reducing memory and computational requirements compared to existing baselines. Our results demonstrate that fully exhausting the representational capacities of pre-trained models in CL goes well beyond their final embeddings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
