SHARP：面向神经启发式持续学习的稀疏性与隐藏激活回放
SHARP: Sparsity and Hidden Activation RePlay for Neuro-Inspired Continual Learning
AI总结：
SHARP提出一种神经启发式持续学习方法，利用稀疏动态连接和激活回放，仅回放最近见过的少数类别即可持续更新所有层，在类别增量学习中优于现有回放方法。
摘要：
深度神经网络（DNNs）在动态环境中学习困难，因为它们依赖固定数据集或静态环境。持续学习（CL）旨在解决这一局限，使DNNs能够像人类学习一样逐步积累知识。受大脑巩固记忆方式的启发，CL中一种强大的策略是回放，即在新的类别和所有已见类别的混合数据上训练DNN。然而，现有回放方法忽略了生物回放的两个关键方面：1）大脑回放的是已处理的神经模式，而不是原始输入；2）大脑优先回放最近学习的信息，而不是重新访问所有过去的经验。为了解决这些差异，我们提出了SHARP，一种高效的神经启发式CL方法，利用稀疏动态连接和激活回放。与其他假设未进行回放的层已经预训练并固定的激活回放方法不同，SHARP能够持续更新所有层。此外，SHARP的独特之处在于它只需要回放少数最近见过的类别，而不是所有过去的类别。我们在五个数据集上的实验表明，SHARP在类别增量学习中优于最先进的回放方法。此外，我们展示了SHARP在一个新的CL场景中的灵活性，其中学习片段之间的边界是模糊的。SHARP代码可在\url{https://github.com/BurakGurbuz97/SHARP-Continual-Learning}获取。
英文摘要：
Deep neural networks (DNNs) struggle to learn in dynamic environments since they rely on fixed datasets or stationary environments. Continual learning (CL) aims to address this limitation and enable DNNs to accumulate knowledge incrementally, similar to human learning. Inspired by how our brain consolidates memories, a powerful strategy in CL is replay, which involves training the DNN on a mixture of new and all seen classes. However, existing replay methods overlook two crucial aspects of biological replay: 1) the brain replays processed neural patterns instead of raw input, and 2) it prioritizes the replay of recently learned information rather than revisiting all past experiences. To address these differences, we propose SHARP, an efficient neuro-inspired CL method that leverages sparse dynamic connectivity and activation replay. Unlike other activation replay methods, which assume layers not subjected to replay have been pretrained and fixed, SHARP can continually update all layers. Also, SHARP is unique in that it only needs to replay few recently seen classes instead of all past classes. Our experiments on five datasets demonstrate that SHARP outperforms state-of-the-art replay methods in class incremental learning. Furthermore, we showcase SHARP's flexibility in a novel CL scenario where the boundaries between learning episodes are blurry. The SHARP code is available at \url{https://github.com/BurakGurbuz97/SHARP-Continual-Learning}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
