通过结构感知掩码和GRPO实现高效的链式推理知识蒸馏课程学习
BRIDGE: Bridging Reasoning In Distillation Gap Elimination via Structure-Aware Masking
AI总结：
通过结构感知掩码和GRPO实现高效的链式推理知识蒸馏，提升学生模型的准确率和输出简洁性。
摘要：
从大语言模型中蒸馏出链式推理（CoT）推理到紧凑的学生模型是一项基本挑战：教师的推理过程通常过于冗长，对于较小的模型难以忠实复现。现有方法要么将推理压缩为单步，从而失去使CoT有价值的可解释性。我们提出一个三阶段的课程学习框架，通过渐进式技能获取来解决这种能力不匹配问题。首先，我们通过掩码洗牌重建建立结构理解。其次，我们对掩码完成任务应用组相对策略优化（GRPO），使模型能够发现自身在准确性和简洁性之间的平衡。第三，我们识别出持续失败案例，并通过定向重写引导学生内化教师知识，再次通过GRPO优化。在GSM8K上的实验表明，我们的方法使Qwen2.5-3B-Base实现了11.29个百分点的准确率提升，同时将输出长度减少了27.4个百分点，超过了指令微调变体和先前的知识蒸馏方法。
英文摘要：
Chain-of-Thought (CoT) reasoning has significantly improved LLMs' mathematical problem-solving capabilities, but distilling such capabilities into smaller models remains challenging due to the capacity mismatch between verbose teachers and compact students. Directly copying teachers' lengthy reasoning chains causes capacity overload, resulting in truncated outputs or repetitive failure. Existing remedies each sacrifice a critical property of CoT: implicit reasoning methods (e.g., compressing reasoning into hidden states) trade away interpretability and verifiability, while heuristic compression strategies (e.g., random step pruning) destroy logical integrity. To address this, we propose BRIDGE, a curriculum framework that first establishes structural understanding via masked reconstruction, then uses GRPO-based reinforcement learning to guide students in self-discovering the optimal balance between accuracy and brevity, and finally internalizes complex reasoning through teacher-guided rewriting on failure cases. On GSM8K, BRIDGE enables Qwen2.5-3B to achieve 11.29% accuracy improvement and 27.4% token reduction over the original model, outperforming instruction-tuned variants and distillation baselines. Zero-shot transfer experiments on SVAMP and MATH-500 further confirm the generalization of internalized reasoning. Our code and model checkpoints are publicly available at https://github.com/Applied-Machine-Learning-Lab/SDM2026_BRIDGE and https://huggingface.co/bowen0815/BRIDGE.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
