FIRE：弗罗贝尼乌斯-等距重初始化以平衡稳定性-可塑性权衡
FIRE: Frobenius-Isometry Reinitialization for Balancing the Stability-Plasticity Tradeoff
AI总结：
FIRE通过量化稳定性与可塑性，平衡神经网络在非平稳数据训练中的稳定性与可塑性权衡，优于传统重初始化方法。
摘要：
深度神经网络在非平稳数据上训练必须平衡稳定性（保留先前知识）和可塑性（适应新任务）。标准重初始化方法（重置权重至原始值）被广泛使用但难以调优：保守的重初始化无法恢复可塑性，而激进的重初始化会擦除有用知识。我们提出FIRE，一种原理性的重初始化方法，明确平衡稳定性-可塑性权衡。FIRE通过平方弗罗贝尼乌斯误差（SFE）量化稳定性，测量与过去权重的接近程度；通过偏离等距（DfI）衡量可塑性，反映权重等距性。重初始化点通过求解约束优化问题获得，最小化SFE同时满足DfI为零，该问题通过牛顿-施卢茨迭代高效近似。FIRE在持续视觉学习（CIFAR-10与ResNet-18）、语言建模（OpenWebText与GPT-0.1B）和强化学习（HumanoidBench与SAC及Atari游戏与DQN）中评估。在所有领域，FIRE均优于无干预训练和标准重初始化方法，证明了有效平衡稳定性-可塑性权衡。
英文摘要：
Deep neural networks trained on nonstationary data must balance stability (i.e., retaining prior knowledge) and plasticity (i.e., adapting to new tasks). Standard reinitialization methods, which reinitialize weights toward their original values, are widely used but difficult to tune: conservative reinitializations fail to restore plasticity, while aggressive ones erase useful knowledge. We propose FIRE, a principled reinitialization method that explicitly balances the stability-plasticity tradeoff. FIRE quantifies stability through Squared Frobenius Error (SFE), measuring proximity to past weights, and plasticity through Deviation from Isometry (DfI), reflecting weight isotropy. The reinitialization point is obtained by solving a constrained optimization problem, minimizing SFE subject to DfI being zero, which is efficiently approximated by Newton-Schulz iteration. FIRE is evaluated on continual visual learning (CIFAR-10 with ResNet-18), language modeling (OpenWebText with GPT-0.1B), and reinforcement learning (HumanoidBench with SAC and Atari games with DQN). Across all domains, FIRE consistently outperforms both naive training without intervention and standard reinitialization methods, demonstrating effective balancing of the stability-plasticity tradeoff.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
