HEAL: 通过回顾熵辅助学习进行推理蒸馏
HEAL: Hindsight Entropy-Assisted Learning for Reasoning Distillation
AI总结：
HEAL通过引入熵辅助学习和三阶段蒸馏策略，有效提升推理蒸馏效果，优于传统方法。
摘要：
从大型推理模型（LRMs）中蒸馏出推理能力到较小的模型通常受到拒绝采样限制。标准方法将教师视为静态过滤器，在教师无法独立探索有效解决方案的复杂
英文摘要：
Distilling reasoning capabilities from Large Reasoning Models (LRMs) into smaller models is typically constrained by the limitations of rejection sampling. Standard methods treat the teacher as a static filter, discarding complex "corner-case" problems where the teacher fails to explore valid solutions independently, thereby creating an artificial "Teacher Ceiling" for the student. In this work, we propose Hindsight Entropy-Assisted Learning (HEAL), an RL-free framework designed to bridge this reasoning gap. Drawing on the educational theory of the Zone of Proximal Development (ZPD), HEAL synergizes three core modules: (1) Guided Entropy-Assisted Repair (GEAR), an active intervention mechanism that detects critical reasoning breakpoints via entropy dynamics and injects targeted hindsight hints to repair broken trajectories; (2) Perplexity-Uncertainty Ratio Estimator (PURE), a ratio-based filtering heuristic that reduces high-anomaly shortcut-like rationales; and (3) Progressive Answer-guided Curriculum Evolution (PACE), a three-stage distillation strategy that organizes training from foundational alignment to hard-case adaptation. Extensive experiments on multiple benchmarks demonstrate that HEAL significantly outperforms traditional SFT distillation and other baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
