arXiv 2606.178892026-06-17cs.LGcs.AIcs.NE

维度控制模块化在持续学习中的有效性

Dimensionality Controls When Modularity Helps in Continual Learning

IT University of Copenhagen, Denmark（丹麦技术大学）；Hasso Plattner Institute, University of Potsdam, Germany（波茨坦大学哈asso 印度学院）

Kathrin Korte, Christian Medeiros Adriano, Joachim Winther Pedersen, Eleni Nisioti, Sebastian Risi

备注：: Accepted to the 2nd Workshop on Compositional Learning (CompLearn) at ICML 2026, Seoul, South Korea. 8 pages, 5 figures
发表信息：: Poster, 2nd ICML Workshop on Compositional Learning: Safety, Interpretability, and Agents (CompLearn), ICML 2026

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

研究在持续学习中，模块化架构、任务相似性和表示维度如何共同影响组合学习，发现低维“丰富”机制下模块化结构显著提升性能，而高维“懒惰”机制下影响较小。

摘要：

组合学习系统必须平衡可塑性（获取新知识的能力）与稳定性（保留先前学习组件的能力），尤其是当任务共享结构并存在干扰风险时。我们研究了模块化架构、任务相似性和表示维度如何在顺序A-B-A范式中共同塑造组合持续学习，通过权重尺度操作诱导高维和低维机制，比较了任务分区循环网络与单网络基线。在高维“懒惰”机制中，两种架构实现了相似的性能和内部几何结构，表明当表示受到弱约束时，显式模块化结构影响甚微。在低维“丰富”机制中，模块化变得决定性：模块化网络发展出分级的任务特定子空间，这些子空间在相似任务上重叠，在中等不相似任务上部分对齐，在不相似任务上分离，从而产生比单网络更具组合性和可解释性的组织。这些发现表明，由初始化尺度诱导的表示机制（与表示维度共变）是决定组合性模块化结构在持续学习中何时功能有益的关键因素，并支持将安全性和鲁棒性视为表示子空间的自适应分配问题，而非固定分离或共享。

英文摘要：

Compositional learning systems must balance plasticity, the ability to acquire new knowledge, with stability, the preservation of previously learned components, especially when tasks share structure and risk interference. We study how modular architecture, task similarity, and representational dimensionality jointly shape compositional continual learning in a sequential A-B-A paradigm, comparing a task-partitioned recurrent network to a single-network baseline while inducing high- and low-dimensional regimes via weight-scale manipulations. In a high-dimensional "lazy" regime, both architectures achieve similar performance and internal geometry, suggesting that explicit modular structure has little impact when representations are weakly constrained. In a lower-dimensional "rich" regime, modularity becomes decisive: the modular network develops graded task-specific subspaces that overlap for similar tasks, partially align for moderately dissimilar tasks, and separate for dissimilar tasks, yielding a more compositional and interpretable organization than the single network. These findings identify the representational regime induced by initialization scale, which co-varies with representational dimensionality, as a key factor governing when compositional, modular structure is functionally beneficial in continual learning, and support viewing safety and robustness as problems of adaptive allocation of representational subspaces rather than fixed separation versus sharing.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。