MoRe：模块化表示用于序列数据的原理化持续表示学习
MoRe: Modular Representations for Principled Continual Representation Learning on Sequential Data
AI总结：
本文提出MoRe框架，通过模块化表示方法实现序列数据的原理化持续学习，其核心贡献是通过分解知识为可识别的模块层级，实现模块的重用、对齐和扩展，从而在保持旧模块的同时提升模型的可塑性和稳定性。
摘要：
持续学习要求模型在适应新数据的同时保持已获得的知识。其核心挑战可以视为原理化的一步适应：在最小干扰的情况下将新信息整合到现有表示中。大多数现有方法通过监督、任务特定的方式修改模型参数或架构来解决这一挑战。然而，根本问题在于表示层面：任务需要具有不同但结构化的表示，这些表示可以被选择性更新而不破坏表示，同时结构应反映数据中的内在组织而非任务边界。在序列数据中，时间延迟依赖性提供了一种自然的信号，用于揭示这种组织，展示如何基本表示产生更具体的表示。受人类大脑模块化组织的启发，我们提出MoRe，一个框架，它在表示本身中识别模块性，而不是在架构层面分配。MoRe将知识分解为具有可识别保证的基本和特定模块层级，使在适应过程中能够实现原理化的模块重用、对齐和扩展，同时通过构造保留旧模块。在合成基准和真实世界LLM激活数据上的实验表明了可解释的层次结构，改进了可塑性-稳定性权衡，表明MoRe是持续适应的原理化基础。
英文摘要：
Continual learning requires models to adapt to new data while preserving previously acquired knowledge. At its core, this challenge can be viewed as principled one-step adaptation: incorporating new information with minimal interference to existing representations. Most existing approaches address this challenge by modifying model parameters or architectures in a supervised, task-specific manner. However, the underlying issue is representational: tasks require distinct yet structured representations that can be selectively updated without disrupting representations, while structure should reflect intrinsic organization in the data rather than task boundaries. In sequential data, time-delayed dependencies provide a natural signal for uncovering this organization, revealing how fundamental representations give rise to more specific ones. Inspired by the modular organization of the human brain, we propose MoRe, a framework that identifies modularity in the representation itself rather than allocating it at the architectural level. MoRe decomposes knowledge into a hierarchy of fundamental and specific modules with identifiability guarantees, enabling principled module reuse, alignment, and expansion during adaptation while preserving old modules by construction. Experiments on synthetic benchmarks and real-world LLM activations demonstrate interpretable hierarchical structure, improved plasticity-stability trade-offs, suggesting MoRe as a principled foundation for continual adaptation
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
