BARD: 带预算意识的推理蒸馏
BARD: budget-aware reasoning distillation
AI总结：
BARD通过两阶段训练实现推理能力蒸馏与预算控制，使模型在不同预算下高效完成推理任务。
摘要：
尽管长链式推理（CoT）蒸馏能有效将推理能力转移到较小的语言模型中，但推理过程往往仍显得冗余且计算预算难以控制，导致资源使用效率低下。为了解决这一限制，我们提出了预算感知推理蒸馏（BARD），一种新的框架，同时蒸馏推理能力并实现对推理长度的细粒度控制。BARD将思考预算作为用户指定的控制信号，使模型能够动态平衡推理性能和计算效率。为实现这一概念，BARD引入了两阶段训练计划。第一阶段是在教师生成的长CoT数据上进行监督微调，该数据被压缩到不同的预算级别，从而引导模型理解预算约束。第二阶段则利用强化学习（RL）从奖励信号中考虑推理性能和预算忠实度。将两阶段计划结合起来对于避免策略退化并确保两个目标联合优化至关重要。广泛的实验表明，我们的方法使8B学生模型在具有挑战性的推理基准（AIME24、AIME25、GPQA）上实现了强大的性能，同时在广泛的预算范围内提供了精确且适应性的推理长度控制。
英文摘要：
While long Chain-of-Thought (CoT) distillation effectively transfers reasoning capability to smaller language models, the reasoning process often remains redundant and computational budget uncontrollable, leading to inefficient resource usage. To address this limitation, we propose \textbf{Budget-Aware Reasoning Distillation (BARD)}, a novel framework that simultaneously distills reasoning capability and enables fine-grained control over the reasoning length. BARD uses the thinking budget as a user-specified control signal, allowing the model to dynamically balance reasoning performance and computational efficiency. To achieve this concept, BARD introduces a two-phase training regimen. The first phase, Supervised Fine-Tuning (SFT) on teacher-generated long CoT data compressed to various budget levels, bootstrapping the model's understanding of budget constraints. The second phase leverages Reinforcement Learning (RL) from a reward signal in consideration of reasoning performance and budget fidelity simultaneously. Incorporating the two-phase regimen is crucial to avoiding policy degradation and ensuring that both objectives are optimized jointly. Extensive experiments demonstrate that our method empowers an 8B student model to achieve strong performance on challenging reasoning benchmarks (\textit{AIME24, AIME25, GPQA}) while providing precise and adaptive control over its reasoning length across a wide range of budgets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
