InstructionCP：一种将大型语言模型快速迁移至目标语言的方法
InstructionCP: A fast approach to transfer Large Language Models into target language
AI总结：
提出InstructionCP方法，通过在持续预训练中整合指令标签，使大型语言模型高效迁移至目标语言，同时保留对话与RLHF能力，仅需0.1B数据即可实现。
摘要：
近年来大型语言模型（LLMs）的快速发展主要集中在英语上，导致模型仅用英语回复。为使这些模型适应其他语言，通常采用持续预训练（CP），随后进行监督微调（SFT）以保持对话能力。然而，CP和SFT会降低模型过滤有害内容的能力。我们提出指令持续预训练（InsCP），将指令标签整合到CP过程中，以在获取新语言的同时防止对话熟练度的丧失。我们的实验表明，InsCP保留了对话能力和人类反馈强化学习（RLHF）能力。在语言对齐、可靠性和知识基准上的实证评估证实了InsCP的有效性。值得注意的是，该方法仅需0.1十亿高质量指令跟随数据，从而减少了资源消耗。
英文摘要：
The rapid development of large language models (LLMs) in recent years has largely focused on English, resulting in models that respond exclusively in English. To adapt these models to other languages, continual pre-training (CP) is often employed, followed by supervised fine-tuning (SFT) to maintain conversational abilities. However, CP and SFT can reduce a model's ability to filter harmful content. We propose Instruction Continual Pre-training (InsCP), which integrates instruction tags into the CP process to prevent loss of conversational proficiency while acquiring new languages. Our experiments demonstrate that InsCP retains conversational and Reinforcement Learning from Human Feedback (RLHF) abilities. Empirical evaluations on language alignment, reliability, and knowledge benchmarks confirm the efficacy of InsCP. Notably, this approach requires only 0.1 billion tokens of high-quality instruction-following data, thereby reducing resource consumption.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
