DESIRE：用于无重放持续学习的动态知识整合
DESIRE: Dynamic Knowledge Consolidation for Rehearsal-Free Continual Learning
AI总结：
针对持续学习中预训练数据信息泄露问题，提出基于LoRA的无重放方法DESIRE，通过动态表示整合与决策边界细化解决分类器偏差，在多数据集上实现最优性能及稳定性与可塑性的平衡。
摘要：
持续学习旨在让模型具备像人类一样保留先前所学知识的能力。近期结合参数高效微调的研究通过引入轻量级扩展模块为该领域注入了新活力。然而，现有方法通常忽视了实验数据已用于预训练模型导致的信息泄露问题，一旦在预训练阶段移除这些重复数据，其性能会严重受损。本文提出一种新的基于LoRA的无重放方法DESIRE。该方法在训练时避免施加额外约束以缓解灾难性遗忘，从而最大化新类别的学习。为整合新旧任务知识，提出两个高效后处理模块：一方面仅保留两组LoRA参数用于合并，并提出动态表示整合以校准合并后的特征表示；另一方面提出决策边界细化以解决仅在新类别数据上训练时的分类器偏差。大量实验表明，该方法在多个数据集上达到最先进性能，且在稳定性与可塑性间实现有效平衡。代码将公开。
英文摘要：
Continual learning aims to equip models with the ability to retain previously learned knowledge like a human. Recent work incorporating Parameter-Efficient Fine-Tuning has revitalized the field by introducing lightweight extension modules. However, existing methods usually overlook the issue of information leakage caused by the fact that the experiment data have been used in pre-trained models. Once these duplicate data are removed in the pre-training phase, their performance can be severely affected. In this paper, we propose a new LoRA-based rehearsal-free method named DESIRE. Our method avoids imposing additional constraints during training to mitigate catastrophic forgetting, thereby maximizing the learning of new classes. To integrate knowledge from old and new tasks, we propose two efficient post-processing modules. On the one hand, we retain only two sets of LoRA parameters for merging and propose dynamic representation consolidation to calibrate the merged feature representation. On the other hand, we propose decision boundary refinement to address classifier bias when training solely on new class data. Extensive experiments demonstrate that our method achieves state-of-the-art performance on multiple datasets and strikes an effective balance between stability and plasticity. Our code will be publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
