非干扰权重场：将模型参数视为可扩展的连续函数
Non-Interfering Weight Fields: Treating Model Parameters as a Continuously Extensible Function
AI总结：
本文提出非干扰权重场（NIWF）框架，通过将模型参数视为可扩展的连续函数，解决大型语言模型中的灾难性遗忘问题，实现能力的版本控制与灵活管理。
摘要：
大型语言模型将所有学习到的知识存储在一个固定的权重向量中。教授模型新能力需要修改这些相同的权重，不可避免地会损害之前学到的知识。这种被称为灾难性遗忘的根本限制，已困扰学者数十年。现有方法将权重视为不可变的 artifacts，通过正则化启发式、回放缓冲区或隔离的适配器模块等技术来保护它们。但这些方法都无法提供对遗忘的结构性保证。在本工作中，我们提出了非干扰权重场（NIWF），一种框架，它用一个学习到的函数替代固定的权重范式，该函数可根据连续能力坐标空间的需求生成权重配置。在训练完一个任务后，我们通过快照字段在锚点上的输出来锁定占据的坐标区域，以在所有未来的训练中强制执行功能锁定。我们使用 Mistral-7B 在顺序指令遵循和代码生成任务上验证 NIWF，证明在已承诺的任务上没有遗忘，并在新任务上表现出竞争性的困惑度。该框架引入了软件般的版本控制概念，用于神经网络智能，其中能力可以被承诺、扩展、组合和回滚，而无需重新训练。
英文摘要：
Large language models store all learned knowledge in a single, fixed weight vector. Teaching a model new capabilities requires modifying those same weights, inevitably degrading previously acquired knowledge. This fundamental limitation, known as catastrophic forgetting, has resisted principled solutions for decades. Existing approaches treat weights as immutable artifacts that must be protected through techniques like regularization heuristics, replay buffers, or isolated adapter modules. The problem is none of these provide a structural guarantee against forgetting. In this work, we propose Non-Interfering Weight Fields (NIWF), a framework that replaces the fixed weight paradigm with a learned function that generates weight configurations on demand from a continuous capability coordinate space. After training on a task, we commit the occupied coordinate region by snapshotting the fields outputs on anchor points to enforce a functional lock during all future training. We validate NIWF on sequential instructionfollowing and code generation tasks using Mistral-7B, demonstrating zero forgetting on committed tasks with competitive perplexity on new tasks. The framework introduces the notion of software-like versioning for neural network intelligence, where capabilities can be committed, extended, composed, and rolled back without retraining.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
