通过解释求解程序从大语言模型中蒸馏算法推理能力
Distilling Algorithmic Reasoning from LLMs via Explaining Solution Programs
AI总结：
本研究提出通过让LLM生成问题-求解程序对的解释，用解释对微调小模型Reasoner以蒸馏算法推理能力，该方法在竞赛级编程任务上的求解率优于思维链基线及直接从程序对学习的模型。
摘要：
蒸馏显式思维链推理路径已成为提升大语言模型（LLM）在各类任务中推理能力的有效方法。然而，在应对对当前最优模型构成重大挑战的复杂任务时，该技术往往难以生成能导向正确答案的有效思维链。本研究提出一种利用LLM解释求解的能力来从中蒸馏推理能力的新方法，并将其应用于竞赛级编程挑战求解。具体而言，我们使用LLM为一组<问题，求解程序>对生成解释，再用<问题，解释>对微调一个更小的语言模型（我们称之为Reasoner（推理器）），使其学习算法推理，能够为未见过的问题生成“如何求解”的提示。实验表明，从解释中学习能让Reasoner（推理器）更有效地引导Coder（编码器）实现程序，在竞赛级编程问题上的求解率优于强思维链基线模型，也优于直接从<问题，求解程序>对学习的模型。我们还额外整理了一个CodeContests格式的测试集，包含246道在模型知识截止日期之后发布的更新题目。
英文摘要：
Distilling explicit chain-of-thought reasoning paths has emerged as an effective method for improving the reasoning abilities of large language models (LLMs) across various tasks. However, when tackling complex tasks that pose significant challenges for state-of-the-art models, this technique often struggles to produce effective chains of thought that lead to correct answers. In this work, we propose a novel approach to distill reasoning abilities from LLMs by leveraging their capacity to explain solutions. We apply our method to solving competitive-level programming challenges. More specifically, we employ an LLM to generate explanations for a set of <problem, solution-program> pairs, then use <problem, explanation> pairs to fine-tune a smaller language model, which we refer to as the Reasoner, to learn algorithmic reasoning that can generate "how-to-solve" hints for unseen problems. Our experiments demonstrate that learning from explanations enables the Reasoner to more effectively guide program implementation by a Coder, resulting in higher solve rates than strong chain-of-thought baselines on competitive-level programming problems. It also outperforms models that learn directly from <problem, solution-program> pairs. We curated an additional test set in the CodeContests format, which includes 246 more recent problems posted after the models' knowledge cutoff.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
