Online-LoRA：通过低秩适应实现无任务在线持续学习
Online-LoRA: Task-free Online Continual Learning via Low Rank Adaptation
AI总结：
本文提出Online-LoRA，一种基于低秩适应和在线权重正则化的无任务在线持续学习框架，通过实时微调预训练ViT模型并自动识别数据分布偏移，在多个基准数据集上超越现有方法。
摘要：
灾难性遗忘是在线持续学习（OCL）中的一个重大挑战，尤其是在没有明确任务边界的非平稳数据流中。重放缓冲区固有的内存限制和隐私顾虑加剧了这一挑战。为了解决灾难性遗忘问题，本文提出了Online-LoRA，一种用于无任务在线持续学习的新框架。Online-LoRA允许实时微调预训练的视觉Transformer（ViT）模型，以解决重放缓冲区的局限性并利用预训练模型的性能优势。作为主要贡献，我们的方法提出了一种新颖的在线权重正则化策略，用于识别和巩固重要模型参数。此外，Online-LoRA利用损失值的训练动态来实现数据分布偏移的自动识别。在众多无任务在线持续学习场景和基准数据集（包括CIFAR-100、ImageNet-R、ImageNet-S、CUB-200和CORe50）上的大量实验表明，Online-LoRA能够稳健地适应各种ViT架构，同时实现优于最先进（SOTA）方法的性能。我们的代码将在https://github.com/Christina200/Online-LoRA-official.git公开提供。
英文摘要：
Catastrophic forgetting is a significant challenge in online continual learning (OCL), especially for non-stationary data streams that do not have well-defined task boundaries. This challenge is exacerbated by the memory constraints and privacy concerns inherent in rehearsal buffers. To tackle catastrophic forgetting, in this paper, we introduce Online-LoRA, a novel framework for task-free OCL. Online-LoRA allows to finetune pre-trained Vision Transformer (ViT) models in real-time to address the limitations of rehearsal buffers and leverage pre-trained models' performance benefits. As the main contribution, our approach features a novel online weight regularization strategy to identify and consolidate important model parameters. Moreover, Online-LoRA leverages the training dynamics of loss values to enable the automatic recognition of the data distribution shifts. Extensive experiments across many task-free OCL scenarios and benchmark datasets (including CIFAR-100, ImageNet-R, ImageNet-S, CUB-200 and CORe50) demonstrate that Online-LoRA can be robustly adapted to various ViT architectures, while achieving better performance compared to SOTA methods. Our code will be publicly available at: https://github.com/Christina200/Online-LoRA-official.git.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
