自我增强推理训练：激活小模型中的潜在推理能力以提升推理蒸馏效果
Self-Enhanced Reasoning Training: Activating Latent Reasoning in Small Models for Enhanced Reasoning Distillation
AI总结：
针对小模型（如GPT-2）推理能力弱于大型语言模型的问题，提出自我增强推理训练（SERT），激活小模型采样时的潜在高质量推理路径并用于自训练，实验证明该方法能提升小模型推理蒸馏效果。
摘要：
大型语言模型（LLMs）的快速发展显著增强了其推理能力，使其能够处理日益复杂的任务。然而，这些能力在GPT-2等更小、计算效率更高的模型中往往会减弱。近期研究表明，推理蒸馏可帮助小模型获得推理能力，但大多数现有方法主要聚焦于改进教师模型生成的推理路径。我们的观察发现，即使没有思维链提示，小模型在采样过程中也能生成高质量推理路径，尽管在标准解码策略下这些路径因概率低而常处于潜在状态。为此，我们提出自我增强推理训练（SERT），通过在零样本条件下对过滤后的自生成推理路径进行自训练，激活并利用小模型中的潜在推理能力。以OpenAI的GPT-3.5为教师模型、GPT-2模型为学生模型的实验表明，SERT提升了小模型的推理能力，改善了其推理蒸馏性能。
英文摘要：
The rapid advancement of large language models (LLMs) has significantly enhanced their reasoning abilities, enabling increasingly complex tasks. However, these capabilities often diminish in smaller, more computationally efficient models like GPT-2. Recent research shows that reasoning distillation can help small models acquire reasoning capabilities, but most existing methods focus primarily on improving teacher-generated reasoning paths. Our observations reveal that small models can generate high-quality reasoning paths during sampling, even without chain-of-thought prompting, though these paths are often latent due to their low probability under standard decoding strategies. To address this, we propose Self-Enhanced Reasoning Training (SERT), which activates and leverages latent reasoning capabilities in small models through self-training on filtered, self-generated reasoning paths under zero-shot conditions. Experiments using OpenAI's GPT-3.5 as the teacher model and GPT-2 models as the student models demonstrate that SERT enhances the reasoning abilities of small models, improving their performance in reasoning distillation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
