在持续学习中通过辅助网络实现更好的稳定性-可塑性权衡
Achieving a Better Stability-Plasticity Trade-off via Auxiliary Networks in Continual Learning
AI总结：
本文提出辅助网络持续学习（ANCL）方法，通过引入辅助网络促进可塑性，并以正则化器在可塑性与稳定性间插值，在任务增量与类增量场景中超越强基线，揭示了稳定性-可塑性权衡的基本原理。
摘要：
与人类按顺序学习新任务的自然能力不同，神经网络已知会遭受灾难性遗忘，即在针对新任务进行优化后，模型在旧任务上的性能急剧下降。此后，持续学习（CL）社区提出了多种解决方案，旨在赋予神经网络学习当前任务的能力（可塑性），同时仍能在先前任务上保持高准确率（稳定性）。尽管取得了显著进展，但可塑性-稳定性权衡仍远未得到解决，其潜在机制也知之甚少。在本研究中，我们提出了辅助网络持续学习（ANCL），这是一种新颖的方法，它应用一个额外的辅助网络来促进可塑性，该网络主要关注稳定性。更具体地说，所提出的框架体现为一个正则化器，它自然地在可塑性和稳定性之间进行插值，在任务增量和类增量场景中超越了强基线。通过对ANCL解决方案的广泛分析，我们揭示了稳定性-可塑性权衡之下的一些基本原理。
英文摘要：
In contrast to the natural capabilities of humans to learn new tasks in a sequential fashion, neural networks are known to suffer from catastrophic forgetting, where the model's performances on old tasks drop dramatically after being optimized for a new task. Since then, the continual learning (CL) community has proposed several solutions aiming to equip the neural network with the ability to learn the current task (plasticity) while still achieving high accuracy on the previous tasks (stability). Despite remarkable improvements, the plasticity-stability trade-off is still far from being solved and its underlying mechanism is poorly understood. In this work, we propose Auxiliary Network Continual Learning (ANCL), a novel method that applies an additional auxiliary network which promotes plasticity to the continually learned model which mainly focuses on stability. More concretely, the proposed framework materializes in a regularizer that naturally interpolates between plasticity and stability, surpassing strong baselines on task incremental and class incremental scenarios. Through extensive analyses on ANCL solutions, we identify some essential principles beneath the stability-plasticity trade-off.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
