arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.01152cs.CLcs.AI
收藏

Brainstacks:通过冻结MoE-LoRA堆栈实现跨领域认知能力的持续LLM学习

Brainstacks: Cross-Domain Cognitive Capabilities via Frozen MoE-LoRA Stacks for Continual LLM Learning

Brains Build Research

Mohammad R. Abu Ayyash

AI总结:

Brainstacks通过冻结MoE-LoRA堆栈实现持续多领域微调,利用残差增强突破单堆栈限制,实现跨领域组合。

摘要:

我们提出了Brainstacks,一种模块化架构,用于持续多领域微调大型语言模型,将领域专业知识作为冻结的适配器堆栈叠加在共享冻结基础之上进行推理。五个相互关联的组件:(1)MoE-LoRA结合Shazeer风格的噪声top-2路由,适用于所有七个transformer投影,在QLoRA 4位量化下使用rsLoRA缩放;(2)内循环通过冻结训练堆栈并添加新堆栈实现残差增强;(3)外循环通过课程有序依赖训练序列特定堆栈;(4)通过随机SVD进行空空间投影,约束新堆栈到与先前方向正交的子空间,实现孤立零遗忘;(5)基于结果的sigmoid元路由器在经验发现的领域组合目标上训练,选择性加权堆栈,实现跨领域组合。两个边界实验:(6)PSN预训练在随机初始化模型上;(7)每领域RL(DPO/GRPO)验证与后SFT对齐的兼容性。在TinyLlama-1.1B(4领域,9堆栈)和Gemma 3 12B IT(5领域,10堆栈)上验证,MoE-LoRA比参数匹配的单LoRA快2.5倍,残差增强突破单堆栈上限,路由系统恢复因未受限制堆栈累积破坏的生成质量。核心发现:基于结果的路由器发现领域堆栈编码可转移的认知原语(指令遵循清晰度、数值推理、程序逻辑、思维链结构),而非领域特定知识,尽管这些堆栈中没有医疗数据,医疗提示在97%的情况下路由到聊天+数学堆栈。

英文摘要:

We present Brainstacks, a modular architecture for continual multi-domain fine-tuning of large language models that packages domain expertise as frozen adapter stacks composing additively on a shared frozen base at inference. Five interlocking components: (1) MoE-LoRA with Shazeer-style noisy top-2 routing across all seven transformer projections under QLoRA 4-bit quantization with rsLoRA scaling; (2) an inner loop performing residual boosting by freezing trained stacks and adding new ones; (3) an outer loop training sequential domain-specific stacks with curriculum-ordered dependencies; (4) null-space projection via randomized SVD constraining new stacks to subspaces orthogonal to prior directions, achieving zero forgetting in isolation; (5) an outcome-based sigmoid meta-router trained on empirically discovered domain-combination targets that selectively weights stacks, enabling cross-domain composition. Two boundary experiments: (6) PSN pretraining on a randomly initialized model; (7) per-domain RL (DPO/GRPO) validating compatibility with post-SFT alignment. Validated on TinyLlama-1.1B (4 domains, 9 stacks) and Gemma 3 12B IT (5 domains, 10 stacks), MoE-LoRA achieves 2.5x faster convergence than parameter-matched single LoRA, residual boosting breaks through the single-stack ceiling, and the routed system recovers generation quality destroyed by ungated stack accumulation. The central finding: the outcome-based router discovers that domain stacks encode transferable cognitive primitives (instruction-following clarity, numerical reasoning, procedural logic, chain-of-thought structure) rather than domain-specific knowledge, with medical prompts routing to chat+math stacks in 97% of cases despite zero medical data in those stacks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑