arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2505.18101cs.LG

动态双缓冲与分治策略用于在线持续学习

Dynamic Dual Buffer with Divide-and-Conquer Strategy for Online Continual Learning

Department of Computing, Imperial College London(帝国理工学院计算机系);Department of Bioengineering, Imperial College London(帝国理工学院生物工程系);School of Information and Software Engineering, University of Electronic Science and Technology of China(电子科技大学信息与软件工程学院)

Congren Dai, Huichi Zhou, Jiahao Huang, Zhenxuan Zhang, Fanwen Wang, Yijian Gao, Guang Yang, Fei Ye

AI总结:

本文提出ODEDM框架,通过动态双缓冲与分治策略,有效缓解在线持续学习中的灾难性遗忘问题,实现跨数据集的高性能表现。

摘要:

在线持续学习(OCL)涉及按顺序到达的数据,并特别受到灾难性遗忘的挑战,这会显著损害模型性能。为了解决这个问题,我们介绍了一个新的框架,即在线动态可扩展双记忆(ODEDM),该框架集成了一个短期记忆用于快速记忆和一个长期记忆,后者被划分为子缓冲区,这些缓冲区由聚类原型锚定,以存储多样化的、类别特定的样本以减轻遗忘。我们提出了一种基于K均值的策略用于原型识别,并提出了一种基于最优传输的机制以保留关键样本,优先考虑那些与其对应的原型具有高相似性的样本。这种设计保留了语义丰富的信息。此外,我们提出了一种分治(DAC)优化策略,将记忆更新分解为子问题,从而减少计算开销。ODEDM作为一个即插即用的模块,可以无缝集成到现有的复习方法中。在标准和不平衡的OCL设置下的实验结果表明,ODEDM在多个数据集上始终实现了最先进的性能,相对于DER家族以及最近的方法如VR-MCL和POCL,提供了显著的改进。

英文摘要:

Online Continual Learning (OCL) involves sequentially arriving data and is particularly challenged by catastrophic forgetting, which significantly impairs model performance. To address this issue, we introduce a novel framework, Online Dynamic Expandable Dual Memory (ODEDM), that integrates a short-term memory for fast memory and a long-term memory structured into sub-buffers anchored by cluster prototypes, enabling the storage of diverse and category-specific samples to mitigate forgetting. We propose a novel K-means-based strategy for prototype identification and an optimal transport-based mechanism to retain critical samples, prioritising those exhibiting high similarity to their corresponding prototypes. This design preserves semantically rich information. Additionally, we propose a Divide-and-Conquer (DAC) optimisation strategy that decomposes memory updates into subproblems, thereby reducing computational overhead. ODEDM functions as a plug-and-play module that can be seamlessly integrated with existing rehearsal-based approaches. Experimental results under both standard and imbalanced OCL settings show that ODEDM consistently achieves state-of-the-art performance across multiple datasets, delivering substantial improvements over the DER family as well as recent methods such as VR-MCL and POCL.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑