用于在线数据增量学习的自调节神经发生
Self-Regulated Neurogenesis for Online Data-Incremental Learning
AI总结：
受大脑自调节神经发生机制启发，提出SERENA方法用于在线数据增量学习，通过在过参数化网络中构建并冻结“概念细胞”路径来防止灾难性遗忘，在十个基准上取得SOTA并超越离线学习性能。
摘要：
神经网络在学习任务序列或数据流时常常面临灾难性遗忘的困境，而人类则不同，即使没有明确的提示线索，也能持续学习并巩固新概念。在线数据增量学习旨在模拟这种能力，其仅对每个样本处理一次，且在任何时刻都无法获取任务或数据流提示，因为与假设新类所有数据均随时可用的离线设置相比，这更符合现实情况。然而，现有方法通常依赖于在内存中存储数据子集或扩展初始模型架构，从而导致显著的计算开销。受“自调节神经发生”——大脑为不同功能创建专门区域或回路的机制——的启发，我们提出了一种新方法SERENA，它将每个概念编码在被称为“概念细胞”的专门网络路径中，并集成到单个过参数化网络内。一旦某个概念被学习，其对应的概念细胞就会被冻结，从而有效防止对已获信息的遗忘。此外，我们引入了两个新的持续学习场景，其特征是样本量逐渐变化，能更紧密地反映现实世界条件。实验结果表明，我们的方法不仅在十个基准上确立了新的最先进结果，而且显著超越了离线监督批量学习的性能。代码可在https://github.com/muratonuryildirim/serena获取。
英文摘要：
Neural networks often struggle with catastrophic forgetting when learning sequences of tasks or data streams, unlike humans who can continuously learn and consolidate new concepts even in the absence of explicit cues. Online data-incremental learning seeks to emulate this capability by processing each sample only once, without having access to task or stream cues at any point in time since this is more realistic compared to offline setups, where all data from novel class(es) is assumed to be readily available. However, existing methods typically rely on storing the subsets of data in memory or expanding the initial model architecture, resulting in significant computational overhead. Drawing inspiration from 'self-regulated neurogenesis'-brain's mechanism for creating specialized regions or circuits for distinct functions-we propose a novel approach SERENA which encodes each concept in a specialized network path called 'concept cell', integrated into a single over-parameterized network. Once a concept is learned, its corresponding concept cell is frozen, effectively preventing the forgetting of previously acquired information. Furthermore, we introduce two new continual learning scenarios that more closely reflect real-world conditions, characterized by gradually changing sample sizes. Experimental results show that our method not only establishes new state-of-the-art results across ten benchmarks but also remarkably surpasses offline supervised batch learning performance. The code is available at https://github.com/muratonuryildirim/serena.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
