TriRE：一种用于持续知识保留与提升的多机制学习范式
TriRE: A Multi-Mechanism Learning Paradigm for Continual Knowledge Retention and Promotion
AI总结：
TriRE提出一种受大脑多机制启发的持续学习范式，通过保留突出神经元、巩固知识并促进低活跃神经元，显著减少任务干扰并超越现有方法。
摘要：
持续学习（CL）由于灾难性遗忘（CF）先前学习任务，一直是深度神经网络面临的持续挑战。为缓解CF，已提出多种技术，如权重正则化、经验回放和参数隔离。尽管这些方法相对成功，但这些研究方向主要保持正交，存在若干缺陷，同时错失了竞争策略的优势。相反，大脑通过同时利用多种神经生理过程，包括神经发生、主动遗忘、神经调节、元可塑性、经验回放和情境依赖门控，持续学习、适应并在任务间迁移知识，很少导致CF。受大脑并发利用多种机制的启发，我们提出TriRE，一种新颖的CL范式，它包含为每个任务保留最突出的神经元，修正并巩固当前和过去任务提取的知识，以及通过回退和重新学习积极促进后续任务中不太活跃的神经元。在各种CL设置中，TriRE显著减少了任务干扰，并超越了单独考虑的多种CL方法。
英文摘要：
Continual learning (CL) has remained a persistent challenge for deep neural networks due to catastrophic forgetting (CF) of previously learned tasks. Several techniques such as weight regularization, experience rehearsal, and parameter isolation have been proposed to alleviate CF. Despite their relative success, these research directions have predominantly remained orthogonal and suffer from several shortcomings, while missing out on the advantages of competing strategies. On the contrary, the brain continually learns, accommodates, and transfers knowledge across tasks by simultaneously leveraging several neurophysiological processes, including neurogenesis, active forgetting, neuromodulation, metaplasticity, experience rehearsal, and context-dependent gating, rarely resulting in CF. Inspired by how the brain exploits multiple mechanisms concurrently, we propose TriRE, a novel CL paradigm that encompasses retaining the most prominent neurons for each task, revising and solidifying the extracted knowledge of current and past tasks, and actively promoting less active neurons for subsequent tasks through rewinding and relearning. Across CL settings, TriRE significantly reduces task interference and surpasses different CL approaches considered in isolation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
