arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.26671cs.AI
收藏

NebulaExp-8B:基于全尺度消融研究的经验性后训练流程

NebulaExp-8B: An Empirical Post-Training Pipeline via Full-Scale Ablation Research

ZTE NebulaL0 Post-Training Team(中兴通讯NebulaL0后训练团队)

Qiaobo Hao, Yangqian Wu, Shunyi Wang, Zhongjian Zhang, Ziqun Li, Yayin He, Muqing Li, Chen Zhong

AI总结:

提出NebulaExp透明后训练流程,基于Qwen3-8B-base,通过数据清洗、三阶段SFT和GRPO RL优化指令与推理分支,并探索OPD/MOPD替代RL,在8B模型上实现性能提升。

摘要:

后训练对齐决定了大型语言模型的推理和遵循人类偏好的能力,然而现有大多数工作隐瞒了详细的数据构建、过滤规则和训练配方,这阻碍了社区的可复现性和轻量级模型优化。本文提出了NebulaExp,一个完全透明、基于消融驱动的后训练流程,构建在Qwen3-8B-base之上,涵盖两个正交的模型分支:通用指令模型和复杂推理专用模型。我们整理了一个包含384万条多源SFT样本的原始语料库和一个20万条可验证的RL候选池,并设计了一个端到端的数据处理栈,包括响应蒸馏、多维交叉验证过滤、细粒度难度分级、任务分类和多样性感知采样。对于指令分支,我们的三阶段优化监督微调方法NebulaExp-Ins-SFT将平均基准分数从Qwen3-8B-nothink的55.01提升到60.99。然后GRPO强化学习进一步将平均分数提升到61.85。对于推理分支,中等难度的GRPO RL将平均推理分数从73.88提升到75.17。为了解决RL对任务验证器的依赖,我们系统地研究了单教师和多教师OPD(MOPD):仅使用4K条指令遵循样本,在IFEval上比RL基线高出3.26分,平均总体增益+4.43;MOPD融合了四个领域专家教师,仅用1万条样本,将平均性能比基础模型提升4.18。本报告为8B规模的大语言模型提供了一个完全可复现的经验性后训练配方,并全面剖析了指令遵循、数学推理、代码生成和通用知识之间的能力权衡。

英文摘要:

Post-training alignment determines the reasoning and human preference following capabilities of large language models, yet most existing works withhold detailed data construction, filtering rules and training recipes, which hinders community reproducibility and lightweight model optimization. This work presents NebulaExp, a fully transparent, ablation-driven post-training pipeline built on Qwen3-8B-base, covering two orthogonal model branches: general instruct model and complex reasoning-specialized model. We curate a raw corpus of 3.84M multi-source SFT samples and a 200K verifiable RL candidate pool, and design an end-to-end data processing stack including response distillation, multi-dimensional cross-verification filtering, fine-grained difficulty grading, task classification and diversity-aware sampling. For the Instruct branch, our three-stage optimized supervised fine-tuning approach NebulaExp-Ins-SFT improves the average benchmark score from the 55.01 baseline of Qwen3-8B-nothink to 60.99. GRPO reinforcement learning then further elevates the average score to 61.85. For the Reasoning branch, medium-difficulty GRPO RL improves average reasoning score from 73.88 to 75.17. To address RL's dependency on task verifiers, we systematically investigate single-teacher and multi-teacher OPD (MOPD): utilizing merely 4K instruction-following samples and outperforms RL baseline by 3.26 points on IFEval with +4.43 average overall gain; MOPD fuses four domain-specialist teachers with merely 10K samples, lifting average performance by 4.18 over the base model. This report provides a fully reproducible empirical post-training recipe for 8B-scale LLMs, and comprehensively dissects the capability trade-offs among instruction adherence, mathematical reasoning, code generation and general knowledge.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑