arXiv 2510.160772026-07-28cs.LGcs.AI

用于领域增量学习的持续知识整合LORA

Continual Knowledge Consolidation LORA for Domain Incremental Learning

STEM, University of South Australia（南澳大利亚大学STEM学院）；School of Artificial Intelligence and Computer Science, Nantong University（南通大学人工智能与计算机科学学院）

Naeem Paeedeh, Mahardhika Pratama, Weiping Ding, Jimmy Cao, Wolfgang Mayer, Ryszard Kowalczyk, Ary Shiddiqi

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对领域增量学习中特定任务LoRA存在的问题，本文提出CONEC-LoRA，通过整合任务共享与特定任务的LoRA提取知识，集成随机分类器概念，部署辅助网络优化预测，在多个基准问题上比现有技术有显著优势。

摘要：

领域增量学习（DIL）是持续学习的一个子分支，旨在解决新领域不断出现且无灾难性遗忘的问题。尽管有参数高效微调（PEFT）方法，但先前工作创建的特定任务LoRA忽略了跨任务的共享知识。推理时特定任务LoRA选择不准确会导致准确率大幅下降，现有工作依赖的线性或基于原型的分类器泛化能力欠佳。本文提出持续知识整合低秩适应（CONEC-LoRA）来解决DIL问题。它由任务共享LoRA与特定任务LoRA整合而成，提取共同知识并纳入特定领域知识。与现有方法不同，CONEC-LoRA集成了随机分类器概念，其参数从分布中采样，提高正确分类可能性。还部署辅助网络优化预测推理用的特定任务LoRA，并实现不同深度网络结构概念，各层与局部分类器相连以利用中间表示。该模块集成球生成器损失和变换模块解决合成样本偏差问题。严格实验表明CONEC-LoRA在4个流行基准问题上比现有技术有超5%的优势。

英文摘要：

Domain Incremental Learning (DIL) is a sub-branch of continual learning that aims to address the never-ending arrival of new domains without catastrophic forgetting. Despite the advent of parameter-efficient fine-tuning (PEFT) approaches, prior works create task-specific LoRAs that overlook shared knowledge across tasks. Inaccurate selection of task-specific LoRAs during inference leads to significant drops in accuracy, while existing works rely on linear or prototype-based classifiers, which have suboptimal generalization powers. Our paper proposes continual knowledge consolidation low-rank adaptation (CONEC-LoRA) addressing the DIL problems. CONEC-LoRA is developed from consolidations between task-shared LORA to extract common knowledge and task-specific LORA to incorporate domain-specific knowledge. Unlike existing approaches, CONEC-LoRA integrates the concept of a stochastic classifier whose parameters are sampled from a distribution, thus enhancing the likelihood of correct classifications. Last but not least, an auxiliary network is deployed to optimally predict the task-specific LoRAs for inference and implements the concept of a different-depth network structure in which every layer is connected with a local classifier to leverage intermediate representations. This module integrates the ball-generator loss and transformation module to address the synthetic sample bias problem. Our rigorous experiments demonstrate the advantage of CONEC-LoRA over prior art on 4 popular benchmark problems with over 5% margins.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。