面向无样本持续学习的带正交正则化的可扩展可微分双记忆架构
Expandable and Differentiable Dual Memories with Orthogonal Regularization for Exemplar-free Continual Learning
AI总结：
针对现有持续学习无法利用任务关联、易出现特征学习冲突的问题，提出带正交正则化的可微分双记忆无样本持续学习方法，在三个基准数据集上性能优于14种主流方法，达到新的性能水平。
摘要：
过往的持续学习方法往往迫使神经网络孤立处理序列任务，既无法利用任务间的有益关联，还会导致网络重复学习相似特征或对特征过度分化。为解决该问题，我们提出一种完全可微分、无需样本的可扩展方法，该方法由两个互补的记忆模块构成：其中一个模块学习可跨所有任务复用的通用特征，另一个模块结合共享特征学习每个样本独有的判别性特征。两个记忆模块均支持可微分，因此网络可自主学习每个样本的潜在表征。针对每个任务，记忆调整模块会自适应剪枝关键槽位，以最小幅度扩展容量以适配新概念，同时通过正交正则化强制保留的记忆组件与新学习的记忆组件实现几何分离，避免相互干扰。在CIFAR-10、CIFAR-100和Tiny-ImageNet数据集上的实验表明，所提方法在类增量学习任务上性能优于14种当前最优方法，最终准确率分别达到55.13%、37.24%和30.11%。补充分析证实，通过对知识的有效整合与利用，所提方法可提升序列任务的平均性能，其生成的特征提取结果最接近性能上限，由此在持续学习领域树立了新的里程碑。
英文摘要：
Continual learning methods used to force neural networks to process sequential tasks in isolation, preventing them from leveraging useful inter-task relationships and causing them to repeatedly relearn similar features or overly differentiate them. To address this problem, we propose a fully differentiable, exemplar-free expandable method composed of two complementary memories: One learns common features that can be used across all tasks, and the other combines the shared features to learn discriminative characteristics unique to each sample. Both memories are differentiable so that the network can autonomously learn latent representations for each sample. For each task, the memory adjustment module adaptively prunes critical slots and minimally expands capacity to accommodate new concepts, and orthogonal regularization enforces geometric separation between preserved and newly learned memory components to prevent interference. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet show that the proposed method outperforms 14 state-of-the-art methods for class-incremental learning, achieving final accuracies of 55.13\%, 37.24\%, and 30.11\%, respectively. Additional analysis confirms that, through effective integration and utilization of knowledge, the proposed method can increase average performance across sequential tasks, and it produces feature extraction results closest to the upper bound, thus establishing a new milestone in continual learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
