GLAM：基于分组LoRA适配器合并的大规模高效持续学习
GLAM: Efficient Continual Learning at Scale via Grouped LoRA Adapter Merging
AI总结：
GLAM是基于LoRA适配器合并的类增量持续学习框架，通过分组、剪枝等技术，在50任务序列的视觉基准上实现最高准确率，仅用16-19%可训练参数，训练时间减少63-74%。
摘要：
在现代机器学习系统中，即使处于基础模型时代，随时间持续学习的能力仍是一项重大挑战。尽管大型预训练模型学习到的丰富表示能部分缓解灾难性遗忘，但它们仍难以高效适应不断变化的数据分布。一个关键挑战在于，如何在向大型预训练模型持续添加新知识时，使其在长任务序列上具备可扩展性和计算效率。本研究中，我们提出GLAM，一种基于LoRA适配器合并的简单且有效的类增量持续学习框架。对于每个任务，GLAM训练一个带重要性标量的轻量级低秩适配器，产生的计算开销极小。随后对适配器进行剪枝、重缩放，并按顺序分组，以实现结构化知识复用并限制参数增长。推理时，所有组合为单个模块，确保无论任务数量多少，计算成本恒定。我们在视觉基准上对GLAM进行评估，任务序列最多达50个，远超标准协议。GLAM在所有评估基准中达到最高准确率。相较于基准方法（在所有基准中平均准确率最高），它使用16-19%的可训练参数，并将训练时间减少约63-74%，展现出高效且可扩展的持续学习能力。我们的源代码公开于此https URL。
英文摘要：
The ability to learn continuously over time remains a major challenge for modern machine learning systems, even in the era of Foundation Models. While the rich representations learned by large pre-trained models can partially mitigate catastrophic forgetting, they still struggle to adapt efficiently to evolving data distributions. A key challenge remains, how to continually add new knowledge to a large pretrained model in a way that is scalable and computationally efficient over long task sequences. In this work, we introduce GLAM, a simple and effective framework for class-incremental continual learning based on LoRA adapters merging. For each task, GLAM trains a lightweight low-rank adapter with an importance scalar, incurring minimal computational overhead. Adapters are then pruned, rescaled, and sequentially grouped to enable structured knowledge reuse and limit parameter growth. At inference, all groups are combined into a single module, ensuring constant computational cost regardless of the number of tasks. We evaluate GLAM on vision benchmarks with sequences of up to 50 tasks, significantly extending beyond standard protocols. GLAM achieves the highest accuracy across the evaluated benchmarks. Compared with the baseline attaining the highest average accuracy across benchmarks, it uses 16--19\% of the trainable parameters and reduces training time by approximately 63--74\%, demonstrating efficient and scalable continual learning. Our source code is publicly available at https://github.com/atlas-luiss/GLAM.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
