动态双缓冲与分治策略用于在线持续学习
Dynamic Dual Buffer with Divide-and-Conquer Strategy for Online Continual Learning
AI总结:
本文提出ODEDM框架,通过动态双缓冲与分治策略,有效缓解在线持续学习中的灾难性遗忘问题,实现跨数据集的高性能表现。
摘要:
在线持续学习(OCL)涉及按顺序到达的数据,并特别受到灾难性遗忘的挑战,这会显著损害模型性能。为了解决这个问题,我们介绍了一个新的框架,即在线动态可扩展双记忆(ODEDM),该框架集成了一个短期记忆用于快速记忆和一个长期记忆,后者被划分为子缓冲区,这些缓冲区由聚类原型锚定,以存储多样化的、类别特定的样本以减轻遗忘。我们提出了一种基于K均值的策略用于原型识别,并提出了一种基于最优传输的机制以保留关键样本,优先考虑那些与其对应的原型具有高相似性的样本。这种设计保留了语义丰富的信息。此外,我们提出了一种分治(DAC)优化策略,将记忆更新分解为子问题,从而减少计算开销。ODEDM作为一个即插即用的模块,可以无缝集成到现有的复习方法中。在标准和不平衡的OCL设置下的实验结果表明,ODEDM在多个数据集上始终实现了最先进的性能,相对于DER家族以及最近的方法如VR-MCL和POCL,提供了显著的改进。
英文摘要:
Online Continual Learning (OCL) involves sequentially arriving data and is particularly challenged by catastrophic forgetting, which significantly impairs model performance. To address this issue, we introduce a novel framework, Online Dynamic Expandable Dual Memory (ODEDM), that integrates a short-term memory for fast memory and a long-term memory structured into sub-buffers anchored by cluster prototypes, enabling the storage of diverse and category-specific samples to mitigate forgetting. We propose a novel K-means-based strategy for prototype identification and an optimal transport-based mechanism to retain critical samples, prioritising those exhibiting high similarity to their corresponding prototypes. This design preserves semantically rich information. Additionally, we propose a Divide-and-Conquer (DAC) optimisation strategy that decomposes memory updates into subproblems, thereby reducing computational overhead. ODEDM functions as a plug-and-play module that can be seamlessly integrated with existing rehearsal-based approaches. Experimental results under both standard and imbalanced OCL settings show that ODEDM consistently achieves state-of-the-art performance across multiple datasets, delivering substantial improvements over the DER family as well as recent methods such as VR-MCL and POCL.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。