动态数据混合最大化混合专家模型的指令微调
Dynamic Data Mixing Maximizes Instruction Tuning for Mixture-of-Experts
AI总结：
针对 MoE 指令微调中固定数据混合权重难以适应训练状态的问题，本文提出依据 token 路由表示与数据集间冗余动态调整采样权重，在有限预算下提升两类 MoE 模型的知识、推理及开放式任务表现。
摘要：
Mixture-of-Experts（MoE，混合专家）模型在指令微调中展现出显著能力，尤其是在任务数量扩展时。然而，以往方法只是简单合并所有训练任务（例如创意写作、编程和数学），并采用固定的采样权重，没有考虑随着模型训练状态变化而产生的不同任务重要性。这样一来，最有帮助的数据无法被有效区分，导致模型性能次优。为减少数据集的潜在冗余，我们首次尝试并提出了一种用于 MoE 指令微调的新型动态数据混合方法。具体而言，受 MoE 的 token 路由偏好启发，我们构建数据集级别的表示，随后捕捉数据集之间的细微差异。最后，我们提出根据数据集彼此之间的冗余度动态调整其采样权重，从而在有限训练预算下最大化全局性能。在两个 MoE 模型上的实验结果表明，我们的方法在下游知识与推理任务以及开放式查询上均有效。代码和模型可在 https://github.com/Spico197/MoE-SFT 获取。
英文摘要：
Mixture-of-Experts (MoE) models have shown remarkable capability in instruction tuning, especially when the number of tasks scales. However, previous methods simply merge all training tasks (e.g. creative writing, coding, and mathematics) and apply fixed sampling weights, without considering the importance of different tasks as the model training state changes. In this way, the most helpful data cannot be effectively distinguished, leading to suboptimal model performance. To reduce the potential redundancies of datasets, we make the first attempt and propose a novel dynamic data mixture for MoE instruction tuning. Specifically, inspired by MoE's token routing preference, we build dataset-level representations and then capture the subtle differences among datasets. Finally, we propose to dynamically adjust the sampling weight of datasets by their inter-redundancies, thus maximizing global performance under a limited training budget. The experimental results on two MoE models demonstrate the effectiveness of our approach on both downstream knowledge \& reasoning tasks and open-ended queries. Code and models are available at https://github.com/Spico197/MoE-SFT .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
