C-LoRA：用于预训练模型的持续低秩适配
C-LoRA: Continual Low-Rank Adaptation for Pre-trained Visual Models
AI总结：
针对现有LoRA微调在持续学习中依赖多适配器导致开销大、推理复杂的问题，提出C-LoRA，利用可学习路由矩阵动态管理参数更新，实现高效知识复用与正交性约束，在基准上取得最先进准确率和参数效率，并提供理论见解。
摘要：
低秩适配（LoRA）是一种高效的微调方法，已广泛应用于自然语言处理和计算机视觉等领域。现有的LoRA微调方法在静态环境中表现出色，但在动态学习中却面临挑战，因为它们依赖多个适配器模块，增加了开销并使推理复杂化。我们提出了持续低秩适配（C-LoRA），这是LoRA在持续学习中的一种新颖扩展。C-LoRA使用一个可学习的路由矩阵来动态管理跨任务的参数更新，确保高效复用已学习的子空间，同时强制正交性以最小化干扰和遗忘。与需要为每个任务单独设置适配器的现有方法不同，C-LoRA实现了任务适配的一体化方法，在顺序学习场景中同时实现了可扩展性和参数效率。C-LoRA在基准测试上达到了最先进的准确率和参数效率，同时提供了关于其路由矩阵在保留和迁移知识中作用的理论见解，为持续学习建立了一个可扩展的框架。
英文摘要：
Pre-trained visual models have become fundamental in computer vision, but they face challenges in continual learning scenarios where data and tasks evolve over time. Low-Rank Adaptation (LoRA) offers efficient fine-tuning capabilities but remains limited for such dynamic environments. Standard LoRA cannot distinguish important subspaces, causing critical knowledge to be overwritten in sequential training. Existing approaches address this by dynamically expanding the set of LoRA adapters, either maintaining a growing pool of task-specific modules or merging new adapters into prior ones, at the cost of unbounded parameter growth or increasing inference complexity. We propose Continual Low-Rank Adaptation (C-LoRA), a method that enables a single, shared LoRA adapter to handle sequential tasks without catastrophic forgetting, without requiring any module selection or fusion at inference. The core of C-LoRA is a learnable routing matrix R that explicitly controls how each rank-one subspace contributes to the weight update. This matrix is decomposed into a stability component (R_base), which preserves knowledge from prior tasks, and a plasticity component (R_delta), which drives adaptation to the current task, providing direct control over the stability-plasticity trade-off. We analyze how R governs gradient flow during sequential training, and demonstrate competitive performance across multiple benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
