信息作为结构对齐:连续学习的动力学理论
Information as Structural Alignment: A Dynamical Theory of Continual Learning
AI总结:
本文提出信息构建框架(IBF),通过结构对齐而非存储内容来实现持续学习,展示了在多个领域中优于重放的保留能力。
摘要:
灾难性遗忘并非工程失败,而是将知识作为全局参数叠加存储的数学结果。现有方法如正则化、重放和冻结子网络均在共享参数基质上添加外部机制。本文引入信息构建框架(IBF),基于信息是结构对齐而非存储内容的成就。在IBF中,两个方程 governing 动力学:运动定律驱动配置向更高一致性发展,修改动力学持续变形一致性景观以响应局部差异。记忆、主体性和自我纠正源于这些动力学而非作为单独模块添加。我们首先在透明的二维玩具模型中展示完整生命周期,然后在三个领域验证:受控非平稳世界、由Stockfish独立评估的国际象棋,以及带有冻结ViT编码器的Split-CIFAR-100。在所有三个领域中,IBF在不存储原始数据的情况下实现了优于重放的保留能力。我们观察到在CIFAR-100上几乎零遗忘(BT = -0.004),国际象棋中的正向转移(+38.5 cp),以及在受控领域中比重放少43%的遗忘。在国际象棋中,框架在独立评估下实现了平均行为优势+88.9 ± 2.8 cp,超过MLP和重放基线。
英文摘要:
Catastrophic forgetting is not an engineering failure. It is a mathematical consequence of storing knowledge as global parameter superposition. Existing methods, such as regularization, replay, and frozen subnetworks, add external mechanisms to a shared-parameter substrate. None derives retention from the learning dynamics themselves. This paper introduces the Informational Buildup Framework (IBF), an alternative substrate for continual learning, based on the premise that information is the achievement of structural alignment rather than stored content. In IBF, two equations govern the dynamics: a Law of Motion that drives configuration toward higher coherence, and Modification Dynamics that persistently deform the coherence landscape in response to localized discrepancies. Memory, agency, and self-correction arise from these dynamics rather than being added as separate modules. We first demonstrate the full lifecycle in a transparent two-dimensional toy model, then validate across three domains: a controlled non-stationary world, chess evaluated independently by Stockfish, and Split-CIFAR-100 with a frozen ViT encoder. Across all three, IBF achieves replay-superior retention without storing raw data. We observe near-zero forgetting on CIFAR-100 (BT = -0.004), positive backward transfer in chess (+38.5 cp), and 43% less forgetting than replay in the controlled domain. In chess, the framework achieves a mean behavioral advantage of +88.9 +/- 2.8 cp under independent evaluation, exceeding MLP and replay baselines.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。