arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.06101cs.CL
收藏

代码推理之谷:扩展大语言模型知识蒸馏的规模

The Valley of Code Reasoning: Scaling Knowledge Distillation of Large Language Models

Muyu He, Muhammad Ali Shafique, Anand Kumar, Tsach Mackey, Nazneen Rajani

AI总结:

本研究揭示代码推理蒸馏中的“谷”现象:性能先降后升,且简单问题更有效,输出正确性不影响结果。

摘要:

将具有推理能力的大语言模型(LLM)的思维轨迹蒸馏到较小的模型中已被证明是有效的。然而,关于模型性能如何随蒸馏数据数量扩展的研究却很少。在本工作中,我们研究了在两个小型非推理LLM上蒸馏竞争性编程技能的扩展趋势。我们验证了一个假设,即存在一个“代码推理之谷”:在竞争性编程上的下游性能首先随着数据数量的增加而下降,然后以比对数线性更陡峭的方式稳步上升。在识别出这一趋势后,我们进一步在相同数据上于两个不同的蒸馏阶段对模型进行微调,以基于它们各自的学习阶段得出结论。我们发现,在低数据和中低数据区域的不同阶段,小型模型从较简单的编程问题中获益显著多于从较难的问题中获益。我们还意外地发现,训练数据中输出的正确性对蒸馏结果没有影响。我们的工作代表了在理解代码推理蒸馏的训练动态方面迈出的一步,超越了直觉。

英文摘要:

Distilling the thinking traces of a Large Language Model (LLM) with reasoning capabilities into a smaller model has been proven effective. Yet, there is a scarcity of work done on how model performances scale with the quantity of distillation data. In this work, we study the scaling trend of distilling competitive coding skills on two small non-reasoning LLMs. We validate the hypothesis that there is a $\textit{valley of code reasoning}$: downstream performance on competitive coding first drops as data quantity increases, then it steadily increases in a sharper-than-log-linear fashion. Having identified the trend, we further fine-tune the models at two different distillation stages on the same data to ground conclusions on their respective learning phases. We learn that across stages in the low and medium-low data regimes, small models benefit significantly from easier coding questions than from harder ones. We also find that, surprisingly, the correctness of outputs in training data makes no difference to distillation outcomes. Our work represents a step forward in understanding the training dynamics of code reasoning distillation outside intuition

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑