XFT：通过简单合并向上循环的混合专家模型释放代码指令微调的力量
XFT: Unlocking the Power of Code Instruction Tuning by Simply Merging Upcycled Mixture-of-Experts
AI总结：
XFT通过共享专家机制、路由权重归一化和可学习模型合并，将向上循环的MoE模型编译回稠密模型，以稠密计算开销实现MoE级性能，显著提升小型代码LLM的指令微调效果。
摘要：
我们提出XFT，一种简单而强大的训练方案，通过简单合并向上循环的混合专家（MoE）模型来释放指令微调代码大语言模型（LLMs）的性能极限。虽然普通的稀疏向上循环无法改善指令微调，XFT在稀疏向上循环中引入了一种共享专家机制和一种新颖的路由权重归一化策略，显著提升了指令微调效果。在微调向上循环的MoE模型之后，XFT引入了一种可学习的模型合并机制，将向上循环的MoE模型重新编译回稠密模型，仅以稠密模型的计算开销实现向上循环MoE级别的性能。通过将XFT应用于一个1.3B模型，我们创建了一个新的最先进的小型代码LLM（<3B），在HumanEval和HumanEval+上分别取得67.1和64.6的pass@1。在相同数据和模型架构下，XFT在HumanEval+上将监督微调（SFT）提升了13%，并在MBPP+、MultiPL-E和DS-1000上带来2%到13%的一致提升，证明了其通用性。XFT与Evol-Instruct和OSS-Instruct等现有技术完全正交，为改进代码指令微调开辟了新的维度。代码可在https://github.com/ise-uiuc/xft获取。
英文摘要：
We introduce XFT, a simple yet powerful training scheme, by simply merging upcycled Mixture-of-Experts (MoE) to unleash the performance limit of instruction-tuned code Large Language Models (LLMs). While vanilla sparse upcycling fails to improve instruction tuning, XFT introduces a shared expert mechanism with a novel routing weight normalization strategy into sparse upcycling, which significantly boosts instruction tuning. After fine-tuning the upcycled MoE model, XFT introduces a learnable model merging mechanism to compile the upcycled MoE model back to a dense model, achieving upcycled MoE-level performance with only dense-model compute. By applying XFT to a 1.3B model, we create a new state-of-the-art tiny code LLM (<3B) with 67.1 and 64.6 pass@1 on HumanEval and HumanEval+ respectively. With the same data and model architecture, XFT improves supervised fine-tuning (SFT) by 13% on HumanEval+, along with consistent improvements from 2% to 13% on MBPP+, MultiPL-E, and DS-1000, demonstrating its generalizability. XFT is fully orthogonal to existing techniques such as Evol-Instruct and OSS-Instruct, opening a new dimension for improving code instruction tuning. Codes are available at https://github.com/ise-uiuc/xft.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
