长短思维链混合监督微调激发大语言模型的高效推理能力
Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning Eliciting Efficient Reasoning in Large Language Models
AI总结：
针对大模型监督微调中继承教师模型“过度思考”的问题，提出长短思维链混合监督微调方法，在提升多基准准确率的同时大幅缩短响应长度，实现非推理模型的高效推理。
摘要：
大语言模型的最新研究进展表明，利用从大型推理模型（如DeepSeek R1）蒸馏得到的思维链（Chain-of-Thought, CoT）推理数据进行监督微调（Supervised Fine-Tuning, SFT），可有效将推理能力迁移至非推理模型。但通过该方法微调的模型会继承教师模型的“过度思考”问题，在推理时生成冗长冗余的推理链。为解决这一挑战，我们提出长短思维链混合监督微调（LS-Mixture SFT），将长CoT推理数据集与通过结构保留重写得到的对应短CoT数据集结合使用。实验表明，采用LS-Mixture SFT训练的模型，与直接SFT训练的模型相比，在多个基准测试上平均准确率提升2.3%，同时模型响应长度大幅缩短约47.61%。本研究提供了一种通过监督微调赋予非推理模型推理能力的方法，同时避免了从教师模型继承的固有过度思考问题，从而使微调后的模型实现高效推理。
英文摘要：
Recent advances in large language models have demonstrated that Supervised Fine-Tuning (SFT) with Chain-of-Thought (CoT) reasoning data distilled from large reasoning models (e.g., DeepSeek R1) can effectively transfer reasoning capabilities to non-reasoning models. However, models fine-tuned with this approach inherit the "overthinking" problem from teacher models, producing verbose and redundant reasoning chains during inference. To address this challenge, we propose Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning (LS-Mixture SFT), which combines long CoT reasoning dataset with their short counterparts obtained through structure-preserved rewriting. Our experiments demonstrate that models trained using the LS-Mixture SFT method, compared to those trained with direct SFT, achieved an average accuracy improvement of 2.3% across various benchmarks while substantially reducing model response length by approximately 47.61%. This work offers an approach to endow non-reasoning models with reasoning capabilities through supervised fine-tuning while avoiding the inherent overthinking problems inherited from teacher models, thereby enabling efficient reasoning in the fine-tuned models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
