Unveiling the Impact of Coding Data Instruction Fine-Tuning on Large Language Models Reasoning
University of California, Santa Barbara(加州大学圣巴巴拉分校);The University of Texas at Dallas(德克萨斯大学达拉斯分校);The University of Texas at Austin(德克萨斯大学奥斯汀分校);University of Maryland, College Park(马里兰大学帕克分校)
Xinlu Zhang, Zhiyu Zoey Chen, Xi Ye, Xianjun Yang, Lichang Chen, William Yang Wang, Linda Ruth Petzold
指令微调(Instruction Fine-Tuning, IFT)可显著提升预训练大语言模型(Large Language Models, LLMs)的零样本能力。尽管已知编码数据在预训练阶段能增强LLM的推理能力,但其在IFT阶段激活内部推理能力的作用仍缺乏充分研究。本文探讨了一个核心问题:在IFT阶段,编码数据如何影响LLM的推理能力?为探究该问题,我们从多个维度全面考察了不同编码数据占比、模型家族、模型规模和推理领域下编码数据的影响。具体而言,我们构建了三个编码数据占比依次递增的IFT数据集,在这些数据集上微调了六个来自不同家族和规模的LLM骨干模型,在三个推理领域的十二项任务上评估了微调后模型的性能,并从整体、领域级和任务特定三个由宏观到微观的维度分析了结果。我们的整体分析为每个维度提供了有价值的洞见:首先,编码数据微调可提升不同模型家族和规模的LLM的整体推理能力;其次,尽管编码数据的影响因领域而异,但在同一领域内,不同模型家族和规模下的影响趋势一致;此外,编码数据通常能在不同模型家族间带来相当的任务特定收益,而IFT数据集中的最优占比则取决于具体任务。
英文摘要:
Instruction Fine-Tuning (IFT) significantly enhances the zero-shot capabilities of pretrained Large Language Models (LLMs). While coding data is known to boost LLM reasoning abilities during pretraining, its role in activating internal reasoning capacities during IFT remains understudied. This paper investigates a key question: How does coding data impact LLMs' reasoning capacities during IFT stage? To explore this, we thoroughly examine the impact of coding data across different coding data proportions, model families, sizes, and reasoning domains, from various perspectives. Specifically, we create three IFT datasets with increasing coding data proportions, fine-tune six LLM backbones across different families and scales on these datasets, evaluate the tuned models' performance across twelve tasks in three reasoning domains, and analyze the outcomes from three broad-to-granular perspectives: overall, domain-level, and task-specific. Our holistic analysis provides valuable insights into each perspective. First, coding data tuning enhances the overall reasoning capabilities of LLMs across different model families and scales. Moreover, while the impact of coding data varies by domain, it shows consistent trends within each domain across different model families and scales. Additionally, coding data generally provides comparable task-specific benefits across model families, with optimal proportions in IFT datasets being task-dependent.
Unveiling the Lexical Sensitivity of LLMs: Combinatorial Optimization for Prompt Enhancement
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
Instruction-Tuning Large Language Models with Medical Classification Data Improves ICD Coding of German Tumor Diagnoses
University Medical Center of the Johannes Gutenberg University Mainz(美因茨约翰内斯·古腾堡大学医学中心);University Cancer Center of the Johannes Gutenberg University Mainz(美因茨约翰内斯·古腾堡大学癌症中心)
Is LLM an Overconfident Judge? Unveiling the Capabilities of LLMs in Detecting Offensive Language with Annotation Disagreement
Key Laboratory of Social Computing and Cognitive Intelligence, Dalian University of Technology(大连理工大学社会计算与认知智能重点实验室);School of Computer Science and Technology, Xinjiang Normal University(新疆师范大学计算机科学与技术学院);Social AI Studio, Singapore University of Technology and Design(新加坡科技设计大学社会AI工作室)
Demystifying Instruction Mixing for Fine-tuning Large Language Models
Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);LibrAI;Monash University(莫纳什大学);University of British Columbia(不列颠哥伦比亚大学);The University of Melbourne(墨尔本大学)
Unveiling the Learning Mind of Language Models: A Cognitive Framework and Empirical Study
Thrust of Artificial Intelligence, The Hong Kong University of Science and Technology (Guangzhou), China(香港科学与技术大学人工智能研究所);Department of Computer Science and Engineering, The Hong Kong University of Science and Technology Hong Kong SAR, China(香港科学与技术大学计算机科学与工程系);Microsoft Research Asia(微软亚洲研究院);University of Cambridge(剑桥大学);Microsoft(微软)
Using Instruction-Tuned Large Language Models to Identify Indicators of Vulnerability in Police Incident Narratives
School for Business and Society, University of York(约克大学商业与社会学院);ESRC Vulnerability and Policing Futures Research Centre(经济与社会研究委员会脆弱性与警务未来研究中心);School of Law, University of Leeds(利兹大学法学院)
BigCodeArena: Unveiling More Reliable Human Preferences in Code Generation via Execution
Monash University(墨尔本大学);CSIRO’s Data61(CSIRO的数据61);Purdue University(普渡大学);Independent(独立);HKUST (Guangzhou)(香港科技大学(广州));UCSD(加州大学圣地亚哥分校);UVA(弗吉尼亚大学);CNRS, France(法国国家科学研究中心);IBM;Cisco(思科);Comenius University in Bratislava(布拉提斯拉瓦康门纽斯大学);University of Notre Dame(Notre Dame大学);Uber;Tano Labs(Tano实验室);NUS(国立大学新加坡);Institute of Automation, CAS(中国科学院自动化研究所);Tencent AI Lab(腾讯AI实验室);University of Washington(华盛顿大学);Nevsky Collective(Nevsky集体);ETH Zurich(苏黎世联邦理工学院);Detomo Inc(Detomo公司);University of Oxford(牛津大学);UIUC(伊利诺伊大学香槟分校);Google(谷歌);NVIDIA;Singapore Management University(新加坡管理学院);ServiceNow Research(ServiceNow研究);Hugging Face
National Engineering Research Center for Software Engineering, Peking University(软件工程国家工程研究中心,北京大学);School of Computer Science, Peking University(北京大学计算机学院);PKU-CMCC(Hubei) Joint Research Lab for LLM Industrial Applications(北京大学-CMCC(湖北)大语言模型工业应用联合实验室);School of Health Sciences and Biomedical Engineering, Hebei University of Technology(河北工业大学健康科学与生物医学工程学院)
Negation Blindness in Large Language Models: Unveiling the NO Syndrome in Image Generation
Aligarh Muslim University(阿里格尔穆斯林大学);VIT Bhopal University(VIT博帕尔大学);Nanyang Technological University(南洋理工大学);Technical University of Munich(慕尼黑工业大学);Imperial College London(伦敦帝国学院);Edinburgh Napier University(爱丁堡龙比亚大学)
CodeBoost: Boosting Code LLMs by Squeezing Knowledge from Code Snippets with RL
Nanyang Technological University(南洋理工大学);University of Electronic Science and Technology of China(电子科技大学);Beihang University(北京航空航天大学);Northeastern University, China(中国东北大学)
Supply-Chain Poisoning Attacks Against LLM Coding Agent Skill Ecosystems
Griffith University(格里菲斯大学);The State Information Center(国家信息中心);Nanyang Technological University(南洋理工大学);University of New South Wales(新南威尔士大学);Wake Forest University(维克森林大学)