arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2512.06266cs.CL
收藏

Nanbeige4-3B 技术报告:探索小型语言模型的前沿

Nanbeige4-3B Technical Report: Exploring the Frontier of Small Language Models

Nanbeige LLM Lab(纳布吉LLM实验室)
Chen Yang, Guangyue Peng, Jiaying Zhu, Ran Le, Ruixiang Feng, Tao Zhang, Wei Ruan, Xiaoqi Liu, Xiaoxue Cheng, Xiyun Xu, Yang Song, Yanzipeng Gao, Yiming Jia, Yu… 展开作者

Chen Yang, Guangyue Peng, Jiaying Zhu, Ran Le, Ruixiang Feng, Tao Zhang, Wei Ruan, Xiaoqi Liu, Xiaoxue Cheng, Xiyun Xu, Yang Song, Yanzipeng Gao, Yiming Jia, Yun Xing, Yuntao Wen, Zekai Wang, Zhenwei An, Zhicong Sun, Zongchao Chen

AI总结:

Nanbeige4-3B通过FG-WSD调度器、DPD蒸馏和强化学习技术,实现了小型语言模型在性能和扩展定律上的突破。

摘要:

我们介绍了Nanbeige4-3B,一个小型但表现优异的语言模型家族。在23T高质量token上预训练,并在超过3000万种多样指令上微调,我们扩展了小型语言模型的扩展定律边界。在预训练阶段,我们设计了细粒度渐进稳定衰减(FG-WSD)训练调度器,通过逐步细化各阶段的数据混合来提升模型性能。在后训练阶段,为了提高SFT数据的质量,我们设计了一种联合机制,整合了反思生成优化和推理链重建,从而在复杂任务上取得了显著提升。随后,我们使用旗舰推理模型通过我们提出的双偏好蒸馏(DPD)方法对Nanbeige4-3B进行蒸馏,进一步提升了性能。最后,应用了多阶段强化学习阶段,利用可验证奖励和偏好建模来加强推理和人类对齐能力。广泛评估表明,Nanbeige4-3B不仅显著优于参数规模相当的模型,而且在广泛的基准测试中与更大规模的模型相抗衡。模型检查点可在https://huggingface.co/Nanbeige上获得。

英文摘要:

We present Nanbeige4-3B, a family of small-scale but high-performing language models. Pretrained on 23T high-quality tokens and finetuned on over 30 million diverse instructions, we extend the boundary of the scaling law for small language models. In pre-training, we design a Fine-Grained Warmup-Stable-Decay (FG-WSD) training scheduler, which progressively refines data mixtures across stages to boost model performance. In post-training, to improve the quality of the SFT data, we design a joint mechanism that integrates deliberative generation refinement and chain-of-thought reconstruction, yielding substantial gains on complex tasks. Following SFT, we employ our flagship reasoning model to distill Nanbeige4-3B through our proposed Dual Preference Distillation (DPD) method, which leads to further performance gains. Finally, a multi-stage reinforcement learning phase was applied, leveraging verifiable rewards and preference modeling to strengthen abilities on both reasoning and human alignment. Extensive evaluations show that Nanbeige4-3B not only significantly outperforms models of comparable parameter scale but also rivals much larger models across a wide range of benchmarks. The model checkpoints are available at https://huggingface.co/Nanbeige.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑