Humans can learn several tasks in succession with minimal mutual interference but perform more poorly when trained on multiple tasks at once. The opposite is true for standard deep neural networks. Here, we propose novel computational constraints for artificial neural networks, inspired by earlier work on gating in the primate prefrontal cortex, that capture the cost of interleaved training and allow the network to learn two tasks in sequence without forgetting. We augment standard stochastic gradient descent with two algorithmic motifs, so-called "sluggish" task units and a Hebbian training step that strengthens connections between task units and hidden units that encode task-relevant information. We found that the "sluggish" units introduce a switch-cost during training, which biases representations under interleaved training towards a joint representation that ignores the contextual cue, while the Hebbian step promotes the formation of a gating scheme from task units to the hidden layer that produces orthogonal representations which are perfectly guarded against interference. Validating the model on previously published human behavioural data revealed that it matches performance of participants who had been trained on blocked or interleaved curricula, and that these performance differences were driven by misestimation of the true category boundary.
Hebbian Learning based Orthogonal Projection for Continual Learning of Spiking Neural Networks
School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Research Institute of Big Data（深圳大数据研究院）；School of Public Health, Peking University（北京大学公共卫生学院）；Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）；Peng Cheng Laboratory（鹏城实验室）
Incremental learning for audio classification with Hebbian Deep Neural Networks
Signal Processing Research Centre, Tampere University, Tampere, Finland（塔尔库大学信号处理研究中心，芬兰塔尔库）；Department of Electronics, Information and Bioengineering (DEIB), Politecnico di Milano, Italy（米兰理工学院电子、信息与生物工程系）
Rethinking Hebbian Principle: Low-Dimensional Structural Projection for Unsupervised Learning
School of Computer Science and Engineering, UESTC（UESTC计算机科学与工程学院）；Glasgow College, UESTC（UESTC格拉斯哥学院）；School of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；State Key Lab of Brain-Machine Intelligence, Zhejiang University, China（浙江大学脑机智能国家重点实验室）
Latent-LoRA: Compact Latent-Space Adapters with Gradient-Free Routing for Continual Learning
College of Health Solutions, Arizona State University（亚利桑那州立大学健康解决方案学院）；Department of Electrical and Computer Engineering, The University of Texas at Austin（德克萨斯大学奥斯汀分校电气与计算机工程系）；School of Computing and Augmented Intelligence, Arizona State University（亚利桑那州立大学计算与增强智能学院）；The GAME School, Arizona State University（亚利桑那州立大学GAME学院）
GAP-Prompt: Gated Adaptive Prompting for Efficient Continual Learning
Université d’Orléans（奥尔良大学）；INSA CVL（中央卢瓦尔河谷国立应用科学学院）；CY Cergy Paris University（塞尔吉-蓬图瓦兹大学）；ENSEA（法国高等电子与数字学院）；CNRS（法国国家科学研究中心）；Université de Technologie de Troyes（特鲁瓦技术大学）
LLMs Can Evolve Continually on Modality for X-Modal Reasoning
Dalian University of Technology（大连理工大学）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；Tsinghua University（清华大学）；The Hong Kong University of Science and Technology（香港科技大学）
Rank-Order N-of-M Codes for Sparse Distributed Memory: Disentangling Representation and Learning Effects in Noise Robustness Against Contemporary Neuromorphic Architectures
