arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.00900cs.AIcs.CL
收藏

从单个词到数学:语言模型中数学推理学习动态

From Next-Token to Mathematics: The Learning Dynamics of Mathematical Reasoning in Language Models

Department of Computer Science 1 and Psychology 2(计算机科学系和心理学系);Stanford University(斯坦福大学)

Shubhra Mishra, Gabriel Poesia, Noah D. Goodman

AI总结:

本研究分析了语言模型在预训练和后训练过程中数学推理能力的发展,揭示了数学技能学习顺序与课程设计的相关性,并探讨了指令微调对不同数学技能的影响。

摘要:

大型语言模型(LLMs)仅通过下一个词预测进行训练,就能解决涉及数学推理的广泛问题。但这种能力在训练过程中是如何发展的?我们首次分析了几种开源LLMs在预训练和后训练过程中数学推理能力的发展情况。为此,我们构建了MathCAMPS,一个基于44个细粒度技能的合成数据集,这些技能取自K至8年级的共同核心课程。在一个实验中,我们展示了数学技能在预训练过程中以可测量的方式与人类设计的课程顺序相关联,尽管训练数据是随机排列的。我们还对哪些数学能力受益于指令微调,这是一种广泛使用的后训练方法,以及哪些技能受损进行了详细分析。我们的工作为理解LLM训练动态与推理之间的关系提供了实证基础。

英文摘要:

Large Language Models (LLMs) solely trained on next-token prediction learn to solve a wide range of problems involving mathematical reasoning. But how does this ability evolve during training? We show the first analysis of how mathematical reasoning abilities of several open-weight LLMs develop during pre-training and post-training. To this end, we construct MathCAMPS, a synthetic dataset of novel mathematical reasoning problems grounded in 44 fine-grained skills taken from the Common Core curriculum from K to 8th grades. In one experiment, we show that mathematical skills are learned during pre-training in an order that measurably correlates with the human-designed curriculum, even though training data are randomly ordered. We also show a detailed analysis of which mathematical abilities benefit from instruction tuning, a widely used post-training method and, in contrast, which skills suffer. Our work paves the way for an empirical understanding of LLM training dynamics in relation to reasoning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑