基于沉默突触巩固的类脑持续预训练学习器
Brain-inspired continual pre-trained learner via silent synaptic consolidation
AI总结：
针对预训练模型增量学习的灾难性遗忘问题，受大脑沉默突触机制启发提出Artsy框架，兼顾记忆稳定性与学习可塑性，在类别增量任务上性能优于传统方法且具生物可解释性。
摘要：
预训练模型已展现出出色的泛化能力，但在针对新任务进行增量训练时，仍易遭受灾难性遗忘的影响。现有的基于架构的策略面临两大核心挑战：1）将预训练网络与可训练子网络相结合，使得学习过程中跨不断演化的任务，难以在学习可塑性与记忆稳定性之间达成精妙平衡。2）预训练网络与各类子网络之间缺乏稳固的互连机制，限制了推理阶段对相关信息的有效检索。本研究受成熟大脑中通过脉冲时序依赖可塑性实现的沉默突触激活机制启发，提出Artsy（基于沉默突触巩固的类脑持续学习框架），以提升预训练模型的持续学习能力。Artsy包含两大核心组件：训练阶段，Artsy模拟成熟大脑的动态特性，在预训练网络中维持已学知识的记忆稳定性，同时增强任务特定子网络的学习可塑性。推理阶段，借助突触巩固机制，利用人工沉默突触与功能突触在预训练网络的突触前神经元与子网络的突触后神经元之间建立精准连接，从而实现从测试样本中有效提取相关信息。全面的实验评估表明，在类别增量学习任务上，我们的模型显著优于传统方法，同时也为基于架构的方法提供了更强的生物可解释性。此外，我们认为Artsy为模拟生物突触机制提供了一条有前景的路径，有望推动我们对人工系统与生物系统中神经可塑性的理解。
英文摘要：
Pre-trained models have demonstrated impressive generalization capabilities, yet they remain vulnerable to catastrophic forgetting when incrementally trained on new tasks. Existing architecture-based strategies encounter two primary challenges: 1) Integrating a pre-trained network with a trainable sub-network complicates the delicate balance between learning plasticity and memory stability across evolving tasks during learning. 2) The absence of robust interconnections between pre-trained networks and various sub-networks limits the effective retrieval of pertinent information during inference. In this study, we introduce the Artsy, inspired by the activation mechanisms of silent synapses via spike-timing-dependent plasticity observed in mature brains, to enhance the continual learning capabilities of pre-trained models. The Artsy integrates two key components: During training, the Artsy mimics mature brain dynamics by maintaining memory stability for previously learned knowledge within the pre-trained network while simultaneously promoting learning plasticity in task-specific sub-networks. During inference, artificial silent and functional synapses are utilized to establish precise connections between the pre-synaptic neurons in the pre-trained network and the post-synaptic neurons in the sub-networks, facilitated through synaptic consolidation, thereby enabling effective extraction of relevant information from test samples. Comprehensive experimental evaluations reveal that our model significantly outperforms conventional methods on class-incremental learning tasks, while also providing enhanced biological interpretability for architecture-based approaches. Moreover, we propose that the Artsy offers a promising avenue for simulating biological synaptic mechanisms, potentially advancing our understanding of neural plasticity in both artificial and biological systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
