改进大型语言模型的有监督微调以缓解灾难性遗忘
Improved Supervised Fine-Tuning for Large Language Models to Mitigate Catastrophic Forgetting
AI总结：
提出一种无需原始SFT数据即可缓解大模型灾难性遗忘的微调方法，通过重建指令分布并合成通用数据集与领域数据混合微调，在保留通用能力的同时提升特定任务性能。
摘要：
有监督微调（Supervised Fine-Tuning, SFT）是增强大型语言模型（LLMs）指令遵循能力并使其适应专业领域的关键步骤。然而，SFT通常会导致模型通用能力退化，这种现象被称为灾难性遗忘。当第三方从业者微调开源模型时，这一问题会进一步加剧，因为原始SFT数据通常无法获取。为了应对这一挑战，我们提出了一种新颖且高性价比的SFT方法，该方法在无需访问原始SFT数据的情况下即可有效缓解灾难性遗忘。我们的方法首先重建基础模型可能的指令分布。随后，它采用多模型生成与过滤流水线来合成高质量的通用数据集。该合成数据集与新的领域特定数据混合用于微调。实验结果表明，我们的方法不仅保留了模型在通用领域的能力，还提升了特定任务的性能，优于使用公开可用SFT数据集的基线方法。
英文摘要：
Supervised Fine-Tuning (SFT) is a critical step for enhancing the instruction-following capabilities of Large Language Models (LLMs) and adapting them to specialized domains. However, SFT often leads to a degradation of the model's general abilities, a phenomenon known as catastrophic forgetting. This problem is exacerbated when third-party practitioners fine-tune open-source models, as the original SFT data is typically not available. To address this challenge, we propose a novel and cost-effective SFT method that effectively mitigates catastrophic forgetting without requiring access to the original SFT data. Our approach first reconstructs the likely instruction distribution of the base model. It then employs a multi-model generation and filtering pipeline to synthesize a high-quality general-purpose dataset. This synthetic dataset is mixed with new, domain-specific data for fine-tuning. Experimental results show that our method not only preserves the model's capabilities in general domains but also improves task-specific performance, outperforming baselines that use publicly available SFT datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
