arXiv 2309.162982023-09-30cs.CL

代码数据在哪个训练阶段有助于大语言模型的推理？

At Which Training Stage Does Code Data Help LLMs Reasoning?

National University of Defense Technology（国防科技大学）；Peng Cheng Laboratory（鹏城实验室）；Tsinghua University（清华大学）

Yingwei Ma, Yue Liu, Yue Yu, Yuanliang Zhang, Yu Jiang, Changjian Wang, Shanshan Li

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

该研究系统探究代码数据在预训练、指令微调等不同训练阶段对大语言模型推理能力的影响，通过多领域推理任务验证了混合预训练、指令微调阶段引入代码数据的不同作用及动态混合策略的效果，深化了对LLMs推理能力提升的认知。

摘要：

大语言模型（LLMs）已展现出卓越的推理能力，成为语言技术的基础。受代码数据在LLMs训练中取得的巨大成功启发，我们自然会好奇：在哪个训练阶段引入代码数据才能真正帮助LLMs提升推理能力。为此，本文系统探究了不同训练阶段代码数据对LLMs的影响。具体而言，我们分别在预训练阶段、指令微调阶段以及同时在这两个阶段引入代码数据，随后通过五个领域的六项推理任务，全面且公平地评估了LLMs的推理能力。我们对实验结果进行了严谨分析，并给出了带有洞见的结论。首先，用代码与文本混合数据预训练LLMs，可显著增强LLMs的通用推理能力，且几乎不会对其他任务产生负迁移。此外，在指令微调阶段，代码数据赋予LLMs特定任务的推理能力。再者，代码与文本数据的动态混合策略，可辅助LLMs在训练过程中逐步学习推理能力。这些洞见深化了对LLMs推理能力的理解，可用于其科学问答、法律支持等应用场景。源代码与模型参数已发布于链接：https://github.com/yingweima2022/CodeLLM。

英文摘要：

Large Language Models (LLMs) have exhibited remarkable reasoning capabilities and become the foundation of language technologies. Inspired by the great success of code data in training LLMs, we naturally wonder at which training stage introducing code data can really help LLMs reasoning. To this end, this paper systematically explores the impact of code data on LLMs at different stages. Concretely, we introduce the code data at the pre-training stage, instruction-tuning stage, and both of them, respectively. Then, the reasoning capability of LLMs is comprehensively and fairly evaluated via six reasoning tasks in five domains. We critically analyze the experimental results and provide conclusions with insights. First, pre-training LLMs with the mixture of code and text can significantly enhance LLMs' general reasoning capability almost without negative transfer on other tasks. Besides, at the instruction-tuning stage, code data endows LLMs the task-specific reasoning capability. Moreover, the dynamic mixing strategy of code and text data assists LLMs to learn reasoning capability step-by-step during training. These insights deepen the understanding of LLMs regarding reasoning ability for their application, such as scientific question answering, legal support, etc. The source code and model parameters are released at the link:~\url{https://github.com/yingweima2022/CodeLLM}.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。