何时结构在持续学习中起作用？维度控制了模块化如何塑造表征几何
When Does Structure Matter in Continual Learning? Dimensionality Controls When Modularity Shapes Representational Geometry
AI总结：
本文研究了网络架构、任务相似性和表征维度如何共同影响持续学习中的学习过程，发现高维情况下架构影响较小，而低维情况下模块化结构对任务子空间的对齐和分离具有关键作用。
摘要：
为了保持先前学习的表示，持续学习系统必须在可塑性（获取新知识的能力）和稳定性之间取得平衡。这种稳定性-可塑性困境影响了任务间表示的再利用：共享结构在任务相似时促进迁移，但在新学习干扰现有表示时可能引起干扰。然而，仍不清楚何时以及为何结构分离影响这一权衡。本文研究了网络架构、任务相似性和表征维度如何共同塑造连续任务范式中的学习过程。我们通过系统地改变任务相似性（低、中、高）和权重初始化规模，比较了任务划分的模块化递归网络与单模块基线。我们通过学习表示的有效维度来经验性地表征不同的学习模式。我们发现，在高维情况下，架构影响较小，因为表示足够不受约束，可以容纳多个任务而不会产生强烈干扰。相反，在低维（丰富）情况下，架构分离是决定性的：模块化网络表现出任务特定子空间的渐进对齐（相似任务重叠），部分正交化（中等不相似任务），以及更强的分离（不相似任务）。这种渐进几何在单网络基线中不存在。我们的发现表明，表征维度作为关键组织变量，决定了何时结构分离变得功能相关，并突显了适应性几何作为设计持续学习系统的核心原则。
英文摘要：
To preserve previously learned representations, continual learning systems must strike a balance between plasticity, the ability to acquire new knowledge, and stability. This stability-plasticity dilemma affects how representations can be reused across tasks: shared structure enables transfer when tasks are similar but may also induce interference when new learning disrupts existing representations. However, it remains unclear when and why structural separation influences this trade-off. In this study, we examine how network architecture, task similarity, and representational dimensionality jointly shape learning in a sequential task paradigm inspired by transfer-interference studies. We compare a task-partitioned modular recurrent network with a single-module baseline by systematically varying task similarity (low, medium, high) and the scale of weight initialization, which induces different learning regimes that we empirically characterize through the effective dimensionality of the learned representations. We find that architecture has minimal impact in high-dimensional regimes where representations are sufficiently unconstrained to accommodate multiple tasks without strong interference. In contrast, in lower-dimensional (rich) regimes, architectural separation is decisive: modular networks exhibit graded alignment of task-specific subspaces with overlap for similar tasks, partial orthogonalization for moderately dissimilar tasks, and stronger separation for dissimilar tasks. This graded geometry is absent in the single network baseline. Our findings suggest that representational dimensionality acts as a key organizing variable governing when structural separation becomes functionally relevant, and highlight adaptive geometry as a central principle for designing continual learning systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
