arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2412.08347cs.CLcs.AI
收藏

SmolTulu:更高的学习率与批量大小比值可提升SLM的推理能力

SmolTulu: Higher Learning Rate to Batch Size Ratios Can Lead to Better Reasoning in SLMs

Saudi Data & Artificial Intelligence Authority(沙特数据与人工智能局)

Sultan Alrashed

AI总结:

本研究提出指令微调模型SmolTulu,发现学习率与批量大小的比值对不同任务性能影响有差异,据此优化的模型在20亿参数以下模型的指令跟随与推理任务上达到SOTA,并开源相关资源助力高效模型对齐研究。

摘要:

我们提出了SmolTulu-1.7b-Instruct(本报告中记作SmolTulu-DPO-1130),这是一款经过指令微调的语言模型,它适配了AllenAI的Tulu 3后训练流水线,以增强Huggingface的SmolLM2-1.7B基础模型。通过使用1.35亿参数模型开展的全面实证分析,我们证明学习率与批量大小之间的关系会以依赖任务的方式显著影响模型性能。我们的发现呈现出明确的分化:ARC、GSM8K等推理任务受益于更高的学习率与批量大小比值,而HellaSwag、IFEval等模式识别任务在比值较低时性能最优。这些见解指导了SmolTulu的开发,该模型在20亿参数以下的模型中实现了指令跟随的最先进性能,在IFEval上得分67.7%(提升11%),数学推理方面在GSM8K上得分51.6%(提升3.4%),其另一版本在ARC上得分57.1%(提升5.4%)。我们发布了模型、训练方案和消融研究,以推动高效模型对齐领域的进一步研究,证明了对优化动态的审慎调整有助于缩小小型与大型语言模型之间的能力差距。

英文摘要:

We present SmolTulu-1.7b-Instruct, referenced in this report as SmolTulu-DPO-1130, an instruction-tuned language model that adapts AllenAI's Tulu 3 post-training pipeline to enhance Huggingface's SmolLM2-1.7B base model. Through comprehensive empirical analysis using a 135M parameter model, we demonstrate that the relationship between learning rate and batch size significantly impacts model performance in a task-dependent manner. Our findings reveal a clear split: reasoning tasks like ARC and GSM8K benefit from higher learning rate to batch size ratios, while pattern recognition tasks such as HellaSwag and IFEval show optimal performance with lower ratios. These insights informed the development of SmolTulu, which achieves state-of-the-art performance among sub-2B parameter models on instruction following, scoring 67.7% on IFEval ($Δ$11%), and mathematical reasoning with 51.6% on GSM8K ($Δ$3.4%), with an alternate version achieving scoring 57.1% on ARC ($\Delta5.4%$). We release our model, training recipes, and ablation studies to facilitate further research in efficient model alignment, demonstrating that careful adaptation of optimization dynamics can help bridge the capability gap between small and large language models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑