使用上下文依赖门控和突触稳定减轻灾难性遗忘
Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization
摘要：
人类和大多数动物都可以在不忘记旧任务的情况下学习新任务。然而，在新任务上训练人工神经网络（ANN）通常会导致它忘记以前学习的任务。这种现象是“灾难性遗忘“的结果，在这种情况下，训练ANN会破坏对解决先前任务很重要的连接权重，从而降低任务性能。最近的几项研究提出了稳定人工神经网络连接权重的方法，这些方法被认为对解决任务最重要，这有助于减轻灾难性遗忘。在这里，从被认为是在体内实现的算法的灵感，我们提出了一个补充的方法：添加一个上下文相关的门控信号，这样，只有稀疏的，大部分是不重叠的模式的单位是活跃的任何一个任务。这种方法易于实现，需要很少的计算开销，并允许ANN在与权重稳定相结合时在大量顺序呈现的任务中保持高性能。这项工作提供了另一个例子，说明神经科学启发的算法如何使ANN设计和能力受益。
英文摘要：
Humans and most animals can learn new tasks without forgetting old ones. However, training artificial neural networks (ANNs) on new tasks typically cause it to forget previously learned tasks. This phenomenon is the result of "catastrophic forgetting", in which training an ANN disrupts connection weights that were important for solving previous tasks, degrading task performance. Several recent studies have proposed methods to stabilize connection weights of ANNs that are deemed most important for solving a task, which helps alleviate catastrophic forgetting. Here, drawing inspiration from algorithms that are believed to be implemented in vivo, we propose a complementary method: adding a context-dependent gating signal, such that only sparse, mostly non-overlapping patterns of units are active for any one task. This method is easy to implement, requires little computational overhead, and allows ANNs to maintain high performance across large numbers of sequentially presented tasks when combined with weight stabilization. This work provides another example of how neuroscience-inspired algorithms can benefit ANN design and capability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
