更多编辑，更稳定：理解连续模型编辑中的终身规范化
More Edits, More Stable: Understanding the Lifelong Normalization in Sequential Model Editing
AI总结：
本文研究了连续模型编辑中的终身规范化，提出理论分析揭示其自增强稳定性循环，并通过StableEdit提升长期稳定性。
摘要：
Lifelong Model Editing旨在持续更新大型语言模型中的演变事实，同时保持无关知识和一般能力，但仍然面临灾难性遗忘和模型崩溃的问题。实证研究表明，近期的编辑器在长周期内表现出相同的策略：终身规范化（LN），通过运行统计规范化值梯度。移除LN会导致立即性能崩溃，我们观察到一种反直觉的积极累积效应，早期编辑可以促进未来编辑的成功。然而，LN的机制仍是一个“黑箱”，其在终身稳定性中的精确作用尚不明确。在本文中，我们提供了首次在终身制度下的LN理论分析。我们的分析揭示了一个自我增强的稳定性循环，并证明当与岭正则化回归结合时，LN会产生渐近正交的参数更新和有界的范数，直接缓解遗忘和系统崩溃。基于这些见解，我们推导出StableEdit，通过显式的预热阶段和完全白化强化这一稳定性循环，以最小的开销提高长期稳定性。广泛实验验证了我们的理论并展示了竞争性的性能。我们的代码可在https://github.com/MINE-USTC/StableEdit上获得。
英文摘要：
Lifelong Model Editing aims to continuously update evolving facts in Large Language Models while preserving unrelated knowledge and general capabilities, yet it remains plagued by catastrophic forgetting and model collapse. Empirically, we find that recent editors resilient over long horizons share the same core strategy: Lifelong Normalization (LN), which normalizes value gradients using running statistics. Removing LN causes immediate performance collapse, and we observe a counter-intuitive positive cumulative effect where early edits can promote the success of future edits. Yet the mechanism of LN remains a "black box", leaving its precise role in lifelong stability poorly understood. In this work, we provide the first theoretical account of LN in the lifelong regime. Our analysis reveals a self-reinforcing stability loop and proves that, when combined with ridge-regularized regression, LN yields parameter updates with asymptotic orthogonality and bounded norms, directly mitigating forgetting and systemic collapse. Based on these insights, we derive StableEdit, which strengthens this stability loop via an explicit warm-up stage and full whitening, improving long-horizon stability at minimal overhead. Extensive experiments validate our theory and demonstrate competitive performance. Our code is available at https://github.com/MINE-USTC/StableEdit.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
