利用检索与代码工具将LLM智能体蒸馏到小型模型
Distilling LLM Agent into Small Models with Retrieval and Code Tools
AI总结：
本文提出Agent Distillation框架，通过first-thought prefix提示和自洽动作生成，将LLM智能体的推理与工具使用能力蒸馏到小模型中，在多个任务上取得与更大模型相竞争的性能。
摘要：
大型语言模型（LLMs）在复杂推理任务上表现出色，但计算成本仍然高昂，限制了其实际部署。为解决这一问题，近期工作聚焦于利用教师LLM的思维链（CoT）轨迹将推理能力蒸馏到更小的语言模型（sLMs）中。然而，这种方法在需要稀有事实知识或精确计算的场景中面临困难，sLMs往往因能力有限而产生幻觉。在本工作中，我们提出Agent Distillation框架，该框架不仅转移推理能力，还将基于LLM的智能体的完整任务解决行为转移到配备检索和代码工具的sLMs中。我们沿两个互补方向改进智能体蒸馏：（1）引入一种名为first-thought prefix的提示方法，以提升教师生成轨迹的质量；（2）提出一种自洽动作生成方法，以提高小型智能体在测试时的鲁棒性。我们在事实类和数学类共八个推理任务上评估了我们的方法，涵盖域内和域外泛化。结果表明，小至0.5B、1.5B、3B参数的sLMs能够取得与使用CoT蒸馏微调的更大一档的1.5B、3B、7B模型相竞争的性能，展示了智能体蒸馏在构建实用、使用工具的小型智能体方面的潜力。我们的代码可在https://github.com/Nardien/agent-distillation获取。
英文摘要：
Large language models (LLMs) excel at complex reasoning tasks but remain computationally expensive, limiting their practical deployment. To address this, recent works have focused on distilling reasoning capabilities into smaller language models (sLMs) using chain-of-thought (CoT) traces from teacher LLMs. However, this approach struggles in scenarios requiring rare factual knowledge or precise computation, where sLMs often hallucinate due to limited capability. In this work, we propose Agent Distillation, a framework for transferring not only reasoning capability but full task-solving behavior from LLM-based agents into sLMs with retrieval and code tools. We improve agent distillation along two complementary axes: (1) we introduce a prompting method called first-thought prefix to enhance the quality of teacher-generated trajectories; and (2) we propose a self-consistent action generation for improving test-time robustness of small agents. We evaluate our method on eight reasoning tasks across factual and mathematical domains, covering both in-domain and out-of-domain generalization. Our results show that sLMs as small as 0.5B, 1.5B, 3B parameters can achieve performance competitive with next-tier larger 1.5B, 3B, 7B models fine-tuned using CoT distillation, demonstrating the potential of agent distillation for building practical, tool-using small agents. Our code is available at https://github.com/Nardien/agent-distillation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
