为大型语言模型进行知识蒸馏
Knowledge Distillation for Large Language Models
AI总结：
本文提出一种高效的压缩框架，结合引导的思维链强化学习，通过知识蒸馏和链式思维提示提升模型效率，实现更小规模的模型部署。
摘要：
我们提出了一种资源高效的框架，通过知识蒸馏和引导的思维链强化学习来压缩大型语言模型。使用Qwen 3B作为教师，Qwen 0.5B作为学生，在英语Dolly-15k、西班牙Dolly-15k以及代码BugNet和PyTorrent数据集上应用知识蒸馏，通过在英语设置中调整超参数以优化学生性能。在各种任务中，蒸馏后的学生保留了教师大部分能力，同时显著更小：在英语中70%至91%，在西班牙中高达95%，在代码中高达93.5% Rouge-L。对于编码任务，结合链式思维提示与Group Relative Policy Optimization使用带有CoT注释的Codeforces数据，相比仅知识蒸馏，提高了推理连贯性和解决方案正确性。训练后4位权重量化进一步减少了内存占用和推理延迟。这些结果表明，结合链式思维引导的强化学习的知识蒸馏可以产生紧凑、高效的模型，适用于资源受限的部署环境。
英文摘要：
We propose a resource-efficient framework for compressing large language models through knowledge distillation, combined with guided chain-of-thought reinforcement learning. Using Qwen 3B as the teacher and Qwen 0.5B as the student, we apply knowledge distillation across English Dolly-15k, Spanish Dolly-15k, and code BugNet and PyTorrent datasets, with hyperparameters tuned in the English setting to optimize student performance. Across tasks, the distilled student retains a substantial portion of the teacher's capability while remaining significantly smaller: 70% to 91% in English, up to 95% in Spanish, and up to 93.5% Rouge-L in code. For coding tasks, integrating chain-of-thought prompting with Group Relative Policy Optimization using CoT-annotated Codeforces data improves reasoning coherence and solution correctness compared to knowledge distillation alone. Post-training 4-bit weight quantization further reduces memory footprint and inference latency. These results show that knowledge distillation combined with chain-of-thought guided reinforcement learning can produce compact, efficient models suitable for deployment in resource-constrained settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
