TinyThinker：通过由粗到细的知识内化与自我反思蒸馏推理能力
TinyThinker: Distilling Reasoning through Coarse-to-Fine Knowledge Internalization with Self-Reflection
AI总结：
TinyThinker 提出由粗到细的三阶段知识内化和基于自生成数据的自我反思两阶段训练框架，以解决小模型蒸馏推理时表面模仿的问题，并在常识推理基准上取得优于基线的性能。
摘要：
大型语言模型在多种任务中展现出令人印象深刻的推理能力，这促使人们努力通过生成的推理数据将这些能力蒸馏到更小的模型中。然而，直接在此类合成的推理数据上进行训练可能导致对推理过程的表面模仿，而非促进推理能力与底层知识的真正融合。为解决这一问题，我们提出了 TinyThinker 框架，该框架引入了两种新方法。首先，我们引入了一个三阶段过程，逐步引导学生模型经历推理过程，将知识从粗粒度逐步细化到细粒度。其次，我们开发了一个两阶段训练框架，包括初始的推理获取阶段和随后利用自生成数据的自我反思阶段。在常识推理基准上的实验表明，与基线相比，TinyThinker 取得了更优的性能。消融研究进一步验证了我们框架中每个组件的有效性。我们期望 TinyThinker 能够扩展到其他知识密集型推理任务，为在较小语言模型中发展有效推理能力提供一种替代策略。代码可在 https://github.com/shengminp/TinyThinker 获取。
英文摘要：
Large Language Models exhibit impressive reasoning capabilities across diverse tasks, motivating efforts to distill these capabilities into smaller models through generated reasoning data. However, direct training on such synthesized reasoning data may lead to superficial imitation of reasoning process, rather than fostering a genuine integration of reasoning capabilities with underlying knowledge. To address this, we propose TinyThinker, a framework introducing two novel approaches. First, we introduce a three-stage process that incrementally guides the student model through the reasoning process, progressively refining knowledge from coarse to fine granularity. Second, we develop a two-phase training framework comprising an initial reasoning acquisition phase followed by a self-reflection phase utilizing self-generated data. Experiments on commonsense reasoning benchmarks demonstrate that TinyThinker achieves superior performance compared to baselines. Ablation studies further validate the effectiveness of each component in our framework. We expect that TinyThinker can be extended to other knowledge-intensive reasoning tasks, offering an alternative strategy for developing effective reasoning capabilities in smaller language models. Codes are available at https://github.com/shengminp/TinyThinker
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
