将大语言模型的分解能力蒸馏到紧凑语言模型中
Distilling LLMs' Decomposition Abilities into Compact Language Models
AI总结：
本研究通过离线强化学习将大语言模型的分解能力蒸馏至紧凑模型，并利用AI生成数据集和基线建立，展示了紧凑模型复制复杂问题解决技能的潜力。
摘要：
大语言模型（LLMs）已展现出卓越的推理能力，但其庞大的规模带来了可扩展性挑战，并限制了进一步的定制化。相比之下，紧凑模型虽支持定制化训练，但在解决复杂推理任务时往往表现不足。本研究聚焦于利用离线强化学习将大语言模型的分解技能蒸馏到紧凑模型中。我们利用大语言模型能力的进步来提供反馈，并生成一个专门的任务特定数据集用于训练紧凑模型。开发AI生成的数据集和建立基线构成了我们工作的主要贡献，凸显了紧凑模型在复制复杂问题解决技能方面的潜力。
英文摘要：
Large Language Models (LLMs) have demonstrated proficiency in their reasoning abilities, yet their large size presents scalability challenges and limits any further customization. In contrast, compact models offer customized training but often fall short in solving complex reasoning tasks. This study focuses on distilling the LLMs' decomposition skills into compact models using offline reinforcement learning. We leverage the advancements in the LLM`s capabilities to provide feedback and generate a specialized task-specific dataset for training compact models. The development of an AI-generated dataset and the establishment of baselines constitute the primary contributions of our work, underscoring the potential of compact models in replicating complex problem-solving skills.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
