arXiv 2402.046162024-11-23cs.CLcs.AIcs.LG

超越答案：利用多教师知识蒸馏将推理能力迁移至更小的LLM

Beyond Answers: Transferring Reasoning Capabilities to Smaller LLMs Using Multi-Teacher Knowledge Distillation

University of Notre Dame（圣母大学）；University of Michigan（密歇根大学）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）

Yijun Tian, Yikun Han, Xiusi Chen, Wei Wang, Nitesh V. Chawla

备注：: Accepted by WSDM 2025

AI总结：

本文提出TinyLLM多教师知识蒸馏范式，通过让学生LLM学习多个大型教师LLM的答案与推理依据，并引入上下文示例生成器和teacher-forcing Chain-of-Thought策略，在六个数据集上以更小模型规模显著超越大型教师LLM。

摘要：

将更强的大语言模型（LLMs）的推理能力迁移到更小的模型上一直颇具吸引力，因为较小的LLM部署更灵活且成本更低。在现有解决方案中，知识蒸馏因其出色的效率和泛化能力而脱颖而出。然而，现有方法存在若干缺点，包括知识多样性有限以及缺乏丰富的上下文信息。为了解决这些问题并促进紧凑语言模型的学习，我们提出了TinyLLM，这是一种新的知识蒸馏范式，用于从多个大型教师LLM中学习一个小型学生LLM。特别地，我们鼓励学生LLM不仅生成正确答案，还要理解这些答案背后的推理依据。鉴于不同的LLM拥有多样化的推理技能，我们引导学生模型从多个教师LLM中吸收知识。我们进一步引入了一个上下文示例生成器和一个teacher-forcing Chain-of-Thought策略，以确保推理依据准确且基于合适的上下文场景。在两个推理任务的六个数据集上的大量实验证明了我们方法的优越性。结果表明，尽管模型规模小得多，TinyLLM仍能显著超越大型教师LLM。源代码可在以下地址获取：https://github.com/YikunHan42/TinyLLM。

英文摘要：

Transferring the reasoning capability from stronger large language models (LLMs) to smaller ones has been quite appealing, as smaller LLMs are more flexible to deploy with less expense. Among the existing solutions, knowledge distillation stands out due to its outstanding efficiency and generalization. However, existing methods suffer from several drawbacks, including limited knowledge diversity and the lack of rich contextual information. To solve the problems and facilitate the learning of compact language models, we propose TinyLLM, a new knowledge distillation paradigm to learn a small student LLM from multiple large teacher LLMs. In particular, we encourage the student LLM to not only generate the correct answers but also understand the rationales behind these answers. Given that different LLMs possess diverse reasoning skills, we guide the student model to assimilate knowledge from various teacher LLMs. We further introduce an in-context example generator and a teacher-forcing Chain-of-Thought strategy to ensure that the rationales are accurate and grounded in contextually appropriate scenarios. Extensive experiments on six datasets across two reasoning tasks demonstrate the superiority of our method. Results show that TinyLLM can outperform large teacher LLMs significantly, despite a considerably smaller model size. The source code is available at: https://github.com/YikunHan42/TinyLLM.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。