Intern-S2-Mobius：知识与推理解耦的基础模型
Intern-S2-Mobius: Foundation Model with Decoupled Knowledge and Reasoning
浏览论文内容
中文总结 AI 辅助
该研究提出知识与推理解耦的Mobius-v0架构，基于此构建的7B模型和Intern-S2-Mobius模型，分别在减少训练数据和提升推理速度的同时保持了相近的下游任务性能。
中文摘要 AI 辅助
我们推出Mobius-v0架构，它包含存储知识向量的全局共享Memory（前馈网络FFN），以及多个迭代实现组合推理的Reasoners（自注意力机制Self-Attn）。以隐藏状态为缓存和载体，推理器反复查询内存获取所需知识向量，同时将知识传回推理算子。通过这种知识-推理分离架构，Mobius实现了更优的知识压缩效果和推理效率。基于Mobius-v0架构：1）我们从头训练的7B模型，使用基线模型62.6%的训练数据，取得了与7B Transformer基线相似的下游任务得分；2）我们从Qwen3.5-35B持续预训练得到的Intern-S2-Mobius，取得了相似的下游任务得分，同时实现了近4倍的端到端推理加速。
英文摘要
We introduce Mobius-v0, an architecture that comprises a globally shared Memory (FFN) that stores knowledge vectors and multiple Reasoners (Self-Attn) that iteratively achieve compositional reasoning. Using hidden states as cache and carrier, reasoners repeatedly query memory for required knowledge-vectors, while the knowledge is transmitted back to reasoning operators. Through this knowledge-reasoning-separation architecture, Mobius achieves better knowledge compression and reasoning efficiency. Built upon Mobius-v0 architecture: 1) Our 7B model trained-from-scratch achieves similar downstream score as a 7B Transformer baseline with 62.6% of baseline's training data. 2) Our Intern-S2-Mobius, continually-pretrained from Qwen3.5-35B, achieves similar downstream score while delivering nearly 4x end-to-end inference speedup.
发表机构
- Shanghai AI Laboratory（上海人工智能实验室）
机构由 AI 辅助整理，请以论文原文为准。
