arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2406.15972cs.LGstat.ML

EVCL:带权重巩固的弹性变分持续学习

EVCL: Elastic Variational Continual Learning with Weight Consolidation

University of Oxford(牛津大学)

Hunar Batra, Ronald Clark

AI总结:

针对持续学习中的灾难性遗忘问题,提出融合VCL变分后验近似与EWC正则化参数保护的EVCL混合模型,在5个判别任务的两类增量学习场景中性能均优于现有基线。

摘要:

持续学习旨在让模型学习新任务时不遗忘此前所学内容。本研究提出带权重巩固的弹性变分持续学习(EVCL),这是一种新型混合模型,融合了变分持续学习(VCL)的变分后验近似机制,以及弹性权重巩固(EWC)基于正则化的参数保护策略。通过结合两种方法的优势,EVCL有效缓解了灾难性遗忘,能更好地捕捉模型参数与任务特定数据之间的依赖关系。在5个判别任务上的评估显示,在深度判别模型的领域增量和任务增量学习场景中,EVCL的表现始终优于现有基线方法。

英文摘要:

Continual learning aims to allow models to learn new tasks without forgetting what has been learned before. This work introduces Elastic Variational Continual Learning with Weight Consolidation (EVCL), a novel hybrid model that integrates the variational posterior approximation mechanism of Variational Continual Learning (VCL) with the regularization-based parameter-protection strategy of Elastic Weight Consolidation (EWC). By combining the strengths of both methods, EVCL effectively mitigates catastrophic forgetting and enables better capture of dependencies between model parameters and task-specific data. Evaluated on five discriminative tasks, EVCL consistently outperforms existing baselines in both domain-incremental and task-incremental learning scenarios for deep discriminative models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑