arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2601.20894cs.LG

参数隔离是否更适合基于提示的持续学习?

Is Parameter Isolation Better for Prompt-Based Continual Learning?

Xi’an Jiaotong University(西安交通大学);Shenzhen University of Advanced Technology(深圳大学)

Jiangyang Li, Chenhao Ding, Songlin Dong, Qiang Wang, Jianchao Zhao, Yuhang He, Yihong Gong

AI总结:

本文提出了一种基于提示共享的持续学习框架,通过动态解耦和协作优化提升参数利用效率,有效缓解知识遗忘问题。

摘要:

基于提示的持续学习方法有效缓解了灾难性遗忘问题。然而,大多数现有方法为每个任务分配固定的提示集,完全隔离了任务间的知识,导致参数利用效率低下。为了解决这一问题,我们考虑持续学习的实用需求,提出了一种提示共享框架。该框架构建了一个全局提示池,并引入了任务感知的门控路由机制,通过稀疏激活部分提示实现动态解耦和任务特定特征表示的协作优化。此外,我们引入了一个历史感知调节器,利用累积的提示激活统计信息来保护高频使用的提示免受过度更新,从而缓解无效参数使用和知识遗忘问题。广泛的分析和实验结果表明,我们的方法在有效性和效率方面均优于现有的静态分配策略。

英文摘要:

Prompt-based continual learning methods effectively mitigate catastrophic forgetting. However, most existing methods assign a fixed set of prompts to each task, completely isolating knowledge across tasks and resulting in suboptimal parameter utilization. To address this, we consider the practical needs of continual learning and propose a prompt-sharing framework. This framework constructs a global prompt pool and introduces a task-aware gated routing mechanism that sparsely activates a subset of prompts to achieve dynamic decoupling and collaborative optimization of task-specific feature representations. Furthermore, we introduce a history-aware modulator that leverages cumulative prompt activation statistics to protect frequently used prompts from excessive updates, thereby mitigating inefficient parameter usage and knowledge forgetting. Extensive analysis and empirical results demonstrate that our approach consistently outperforms existing static allocation strategies in effectiveness and efficiency.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑