平衡连续预训练与指令微调：优化LLMs中的指令遵循
Balancing Continuous Pre-Training and Instruction Fine-Tuning: Optimizing Instruction-Following in LLMs
AI总结：
本研究探讨了LLMs连续预训练与指令微调的关系，旨在无需指令数据和微调的情况下，找到一种计算效率最高的策略以同时获取最新知识并保持指令遵循能力。
摘要：
面向公众使用的大型语言模型（LLMs）需要通过连续预训练来保持与最新数据的同步。这些模型还需要使用特定指令进行微调，以维持其准确遵循指令的能力。通常，LLMs会发布两个版本：在多样化数据上预训练的Base LLM，以及额外使用特定指令训练以更好遵循指令的指令精炼LLM。由此产生的问题是：哪个模型应该接受连续预训练，以在保持其指令遵循能力的同时也能跟上最新数据？在本研究中，我们深入探讨了LLMs连续预训练与指令微调之间的复杂关系，并调查了连续预训练对Base模型及其指令微调模型的指令遵循能力的影响。此外，指令微调过程计算量大，且需要大量人工标注样本才能让模型有效学习。本研究旨在找到一种计算效率最高的策略，在不依赖任何指令数据和微调的情况下，获取最新知识并保持指令遵循能力。我们在LLaMa 3、3.1和Qwen 2、2.5系列的Base与instruction模型上实证验证了我们的发现，在不同规模的预训练语料库和不同的LLMs设置下对我们的假设进行了全面探索。
英文摘要：
Large Language Models (LLMs) for public use require continuous pre-training to remain up-to-date with the latest data. The models also need to be fine-tuned with specific instructions to maintain their ability to follow instructions accurately. Typically, LLMs are released in two versions: the Base LLM, pre-trained on diverse data, and the instruction-refined LLM, additionally trained with specific instructions for better instruction following. The question arises as to which model should undergo continuous pre-training to maintain its instruction-following abilities while also staying current with the latest data. In this study, we delve into the intricate relationship between continuous pre-training and instruction fine-tuning of the LLMs and investigate the impact of continuous pre-training on the instruction following abilities of both the base and its instruction finetuned model. Further, the instruction fine-tuning process is computationally intense and requires a substantial number of hand-annotated examples for the model to learn effectively. This study aims to find the most compute-efficient strategy to gain up-to-date knowledge and instruction-following capabilities without requiring any instruction data and fine-tuning. We empirically prove our findings on the LLaMa 3, 3.1 and Qwen 2, 2.5 family of base and instruction models, providing a comprehensive exploration of our hypotheses across varying sizes of pre-training data corpus and different LLMs settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
