更好的开始，更好的结束：用于压缩推理的自引导迭代自推理蒸馏

中文摘要 AI 辅助

大型推理模型常通过长思维链解决问题，但大量计算耗费在冗余推导等上。现有策略内自蒸馏方法存在初始化瓶颈。本文提出BIRD（自引导迭代自推理蒸馏），一种两阶段自推理蒸馏方法。首先在简洁指令下从基础模型采样简洁解，保留正确答案轨迹并执行轻量级提示切换SFT步骤。然后从这个预热模型开始，使用简洁自教师进行策略内反向KL蒸馏。在Qwen3系列模型上，BIRD在MATH - 500和AIME基准测试中比提示和冷启动策略内蒸馏实现了更强的精度 - 效率权衡。如在Qwen3 - 8B上，提高了MATH - 500精度，降低了平均响应长度。结果凸显前缀支持是高效推理蒸馏的核心因素。

英文摘要

Large reasoning models often solve problems through long chain-of-thought (CoT) traces, yet much of this computation is spent on redundant derivations, repeated self-verification, and detours that do not improve the final answer. Existing on-policy self-distillation methods reduce this cost by matching a student model to a concise copy of itself on prefixes sampled from the student's own rollouts. We show that this objective has an initialization bottleneck. Since supervision is applied only to visited prefixes, training from a verbose base model places the KL loss on contexts that are often noisy, redundant, or already off track. In such regions, a concise teacher can provide only local corrections, while the student continues to explore trajectories that an efficient reasoner should avoid. In this paper, we propose BIRD(Bootstrapped Iterative Self-Reasoning Distillation), a two-stage self-reasoning distillation method that improves the rollout distribution before on-policy training. BIRD first samples concise solutions from the base model under a brevity instruction, keeps only answer-correct traces, and performs a lightweight prompt-switch SFT step. The traces are generated with the brevity instruction but learned under the original task prompt, turning instruction-induced conciseness into a default reasoning behavior. Starting from this warm model, BIRD then applies on-policy reverse-KL distillation with a concise self-teacher, now on cleaner and more informative prefixes. Across Qwen3 series models, BIRD achieves a stronger accuracy-efficiency trade-off than prompting and cold-start on-policy distillation on MATH-500 and AIME benchmarks. On Qwen3-8B, it improves MATH-500 accuracy from 86.2% to 92.0% while reducing the average response length from 3,099 to 1,115 tokens. These results highlight prefix support as a central factor in efficient reasoning distillation.

发表机构

• Xi’an Jiaotong University（西安交通大学）
• Peking University（北京大学）

机构由 AI 辅助整理，请以论文原文为准。