NORACL：为无 oracle 的资源自适应持续学习的神经发生
NORACL: Neurogenesis for Oracle-free Resource-Adaptive Continual Learning
AI总结：
NORACL通过生物神经发生机制解决持续学习中的稳定性与可塑性矛盾，动态扩展网络结构以适应不同任务需求，实现更高效的持续学习性能。
摘要：
在持续学习场景中，需要模型既具备学习新任务的可塑性，又保持对先前能力的稳定性。我们认为这一矛盾源于架构层面。有限的网络具有有限的表示和可塑性资源，但所需容量取决于未来任务流的特性，这些特性未知：将遇到多少任务，以及它们在特征空间中的重叠程度。基于正则化的方 法在固定容量架构中保留过去知识，因此隐式依赖于一个为未知未来量身定制的 oracle 架构。当任务仅弱相关时，固定架构逐渐耗尽可塑性资源；当任务较少或强烈重叠时，模型往往过度配置。受生物学中神经发生启发，我们提出 NORACL 来解决稳定性与可塑性矛盾，通过神经元生长来解决 oracle 架构问题。从紧凑网络开始，NORACL 仅在需要时扩展，通过监控两个互补信号来检测表示和可塑性的饱和。我们评估了 NORACL 在不同任务数量和几何结构下的 oracle 量身定制的静态基线。在所有设置中，NORACL 实现的最终平均准确率优于或与 oracle 配置的静态基线相当，同时使用更少的参数。此外，NORACL 产生具有可解释性扩展的架构，即相似任务主要扩展特征提取层，而依赖共同特征的任务则将扩展转向后期特征组合层。我们的分析进一步解释了为什么固定容量网络在任务积累时失去可塑性，而 NORACL 通过生长为新任务创建新的容量。这些结果表明，适应性神经发生推动了持续学习的稳定性-可塑性帕累托前沿。
英文摘要：
In a continual learning setting, we require a model to be plastic enough to learn a new task and stable enough to not disturb previously learned capabilities. We argue that this dilemma has an architectural root. A finite network has limited representational and plastic resources, yet the required capacity depends on properties of the future task stream that are unknown: how many tasks will be encountered, and how much they overlap in feature space. Regularization-based methods preserve past knowledge within fixed-capacity architectures and therefore implicitly rely on an oracle architecture sized for this unknown future. When tasks are only weakly related, fixed architectures progressively run out of plastic resources; when tasks are few or strongly overlapping, models are often over-provisioned. Inspired by neurogenesis in biology, we propose NORACL to address the stability-plasticity dilemma by tackling the oracle architecture problem through neuronal growth. Starting from a compact network, NORACL grows only when needed by monitoring two complementary signals for representational and plasticity saturation. We evaluate NORACL against oracle-sized static baselines across varying task counts and geometries. Across all settings, NORACL achieves final average accuracies that are better than or on par with oracle-provisioned static baselines while using fewer parameters. Additionally, NORACL yields architectures with interpretable growth, i.e. dissimilar tasks predominantly expand feature-extraction layers, whereas tasks which rely on common features shift growth toward later feature-combination layers. Our analysis further explains why fixed-capacity networks lose plasticity as tasks accumulate, whereas NORACL creates fresh capacity for new tasks through growth. Together, these results show that adaptive neurogenesis pushes the stability-plasticity Pareto frontier of continual learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
