arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

SoftModel:一种自行扩展拓扑的神经模型——面向持续服务中学习的受控结构增长

SoftModel: A Neural Model That Grows Its Own Topology -- Governed Structural Growth for Continual In-Service Learning

Zhoumin Xie

arXiv 2608.16409首次发表:更新:

AI 中文总结

该研究提出了软模型(SoftModel),其结构可随非平稳数据流持续受控增长,在标准持续学习基准上保留了长任务序列的持续学习能力,为终身学习提供了新方案。

AI 中文摘要

如今,神经网络系统几乎总是处于两个阶段:训练,然后部署,在这种机制下它会两次被冻结:训练结束时,且拓扑结构本身从未成为自由度。我们将完全可塑性作为公理,即模型的任何部分,包括其结构,都永远不会被冻结,并由此推导出终身学习所需的控制机制。该设计的目标机制是持续服务中学习:针对非平稳数据流的长期模型,其稳定性来自控制而非固定不变,且容量随需求变化。最终得到的是可扩展的软模型:一种结构算子代数(宽度、层级、组合、输入接口、增长的循环、注意力头),每个算子在应用时都是精确的、预算可控且可审计的,其采用仅由保留的现实门决定,该门对参数和结构变化一视同仁。一个完整的从头实现系统由生产型大语言模型(LLM)端到端操作其生成接口。从该公理可得出两个构造性结论:终身变化下的稳定性成为生命周期的审计属性,随需求变化的结构消除了固定拓扑在容量下限约束时对后续能力的隐性限制。第三个结论经测量:在测试场景中,新容量的边际价值在采用前不可观测,因此可行的增长控制采用事后形式。相同的控制机制也适用于评估信号,核心方法在标准持续学习基准上进行评估,受控增长保留了在长任务序列中持续学习的能力。一个预注册的实验项目对测试问题的机制和价值主张进行裁决,并全面公开报告其失败情况;该正负结果映射即为贡献。

英文摘要

Today, a neural system is almost always used in two phases -- trained, then deployed -- and in that regime it freezes twice: training ends, and the topology itself was never a degree of freedom. We take the opposite premise as an axiom -- total plasticity: no part of a model, including its structure, is ever frozen -- and derive the governance a lifelong learner then requires. The design's target regime is continual, in-service learning: a long-lived model on a non-stationary stream, whose stability comes from governance rather than immobility and whose capacity follows demand. The result is a growable soft model: an algebra of structural operators (width, hierarchy, composition, input interface, grown cycles, attention heads), each exact at application, budgeted, and audited, with adoption decided solely by a held-out reality gate that treats parametric and structural change uniformly. A complete from-scratch system realizes the whole account; its factory surface is operated end-to-end by a production LLM. Two conclusions follow from the axiom by construction: stability under lifelong change becomes an audit property of the lifecycle, and structure that follows demand removes the silent cap a fixed topology places on later capability where the capacity floor binds. A third is measured: in the worlds where this was measured, the marginal value of new capacity was unobservable before adoption, so workable growth governance took its ex-post form. The same governance extends to evaluative signals, and the core method is evaluated on standard continual-learning benchmarks, where governed growth preserves the ability to keep learning along long task sequences. A pre-registered experimental program adjudicates the mechanism and value claims on the tested problems and reports its failures at full prominence; the map -- positive and negative -- is the contribution.

Comments99 pages, 16 figures, 12 tables

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑