arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.20535cs.AIcs.CL
收藏

揭示编码数据指令微调对大语言模型推理能力的影响

Unveiling the Impact of Coding Data Instruction Fine-Tuning on Large Language Models Reasoning

University of California, Santa Barbara(加州大学圣巴巴拉分校);The University of Texas at Dallas(德克萨斯大学达拉斯分校);The University of Texas at Austin(德克萨斯大学奥斯汀分校);University of Maryland, College Park(马里兰大学帕克分校)

Xinlu Zhang, Zhiyu Zoey Chen, Xi Ye, Xianjun Yang, Lichang Chen, William Yang Wang, Linda Ruth Petzold

AI总结:

本研究通过构建不同编码数据占比的指令微调数据集,微调多家族多规模大语言模型并在多推理任务上评估,揭示了编码数据在指令微调阶段对模型推理能力的影响规律。

摘要:

指令微调(Instruction Fine-Tuning, IFT)可显著提升预训练大语言模型(Large Language Models, LLMs)的零样本能力。尽管已知编码数据在预训练阶段能增强LLM的推理能力,但其在IFT阶段激活内部推理能力的作用仍缺乏充分研究。本文探讨了一个核心问题:在IFT阶段,编码数据如何影响LLM的推理能力?为探究该问题,我们从多个维度全面考察了不同编码数据占比、模型家族、模型规模和推理领域下编码数据的影响。具体而言,我们构建了三个编码数据占比依次递增的IFT数据集,在这些数据集上微调了六个来自不同家族和规模的LLM骨干模型,在三个推理领域的十二项任务上评估了微调后模型的性能,并从整体、领域级和任务特定三个由宏观到微观的维度分析了结果。我们的整体分析为每个维度提供了有价值的洞见:首先,编码数据微调可提升不同模型家族和规模的LLM的整体推理能力;其次,尽管编码数据的影响因领域而异,但在同一领域内,不同模型家族和规模下的影响趋势一致;此外,编码数据通常能在不同模型家族间带来相当的任务特定收益,而IFT数据集中的最优占比则取决于具体任务。

英文摘要:

Instruction Fine-Tuning (IFT) significantly enhances the zero-shot capabilities of pretrained Large Language Models (LLMs). While coding data is known to boost LLM reasoning abilities during pretraining, its role in activating internal reasoning capacities during IFT remains understudied. This paper investigates a key question: How does coding data impact LLMs' reasoning capacities during IFT stage? To explore this, we thoroughly examine the impact of coding data across different coding data proportions, model families, sizes, and reasoning domains, from various perspectives. Specifically, we create three IFT datasets with increasing coding data proportions, fine-tune six LLM backbones across different families and scales on these datasets, evaluate the tuned models' performance across twelve tasks in three reasoning domains, and analyze the outcomes from three broad-to-granular perspectives: overall, domain-level, and task-specific. Our holistic analysis provides valuable insights into each perspective. First, coding data tuning enhances the overall reasoning capabilities of LLMs across different model families and scales. Moreover, while the impact of coding data varies by domain, it shows consistent trends within each domain across different model families and scales. Additionally, coding data generally provides comparable task-specific benefits across model families, with optimal proportions in IFT datasets being task-dependent.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑