利用生物启发架构提升持续学习任务中的性能
Improving Performance in Continual Learning Tasks using Bio-Inspired Architectures
AI总结：
本文提出一种融合突触可塑性和神经调制的生物启发轻量级架构，通过局部误差信号实现无随机梯度下降的在线持续学习，在多个Split数据集上优于记忆约束方法并达到记忆密集型重放方法水平，同时可提升其他持续学习算法的准确率。
摘要：
从连续到达的数据流中持续学习而不发生灾难性遗忘，对于设计智能系统至关重要。许多持续学习方法依赖随机梯度下降及其变体，这些方法采用全局误差更新，因此需要采取记忆缓冲区或重放等策略来规避其稳定性、贪婪性和短期记忆方面的局限。为了解决这一局限，我们开发了一种受生物启发的轻量级神经网络架构，该架构融合了突触可塑性机制和神经调制，从而通过局部误差信号进行学习，实现在不使用随机梯度下降的情况下进行在线持续学习。与其他受记忆约束的学习方法相比，我们的方法在Split-MNIST、Split-CIFAR-10和Split-CIFAR-100数据集上取得了更优的在线持续学习性能，并且与最先进的记忆密集型重放方法性能相当。我们进一步通过将关键设计概念整合到其他基于反向传播的持续学习算法中，显著提高了这些算法的准确率，从而证明了我们方法的有效性。我们的结果为将生物学原理融入机器学习模型的重要性提供了有力证据，并为如何利用这些原理设计更高效、更鲁棒的在线持续学习系统提供了见解。
英文摘要：
The ability to learn continuously from an incoming data stream without catastrophic forgetting is critical to designing intelligent systems. Many approaches to continual learning rely on stochastic gradient descent and its variants that employ global error updates, and hence need to adopt strategies such as memory buffers or replay to circumvent its stability, greed, and short-term memory limitations. To address this limitation, we have developed a biologically inspired lightweight neural network architecture that incorporates synaptic plasticity mechanisms and neuromodulation and hence learns through local error signals to enable online continual learning without stochastic gradient descent. Our approach leads to superior online continual learning performance on Split-MNIST, Split-CIFAR-10, and Split-CIFAR-100 datasets compared to other memory-constrained learning approaches and matches that of the state-of-the-art memory-intensive replay-based approaches. We further demonstrate the effectiveness of our approach by integrating key design concepts into other backpropagation-based continual learning algorithms, significantly improving their accuracy. Our results provide compelling evidence for the importance of incorporating biological principles into machine learning models and offer insights into how we can leverage them to design more efficient and robust systems for online continual learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
