arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.19405cs.AI
收藏

逻辑蒸馏:面向决策任务按函数逐段从代码中学习

Logic Distillation: Learning from Code Function by Function for Decision-making Tasks

The School of Computer and Artificial Intelligence of Zhengzhou University(郑州大学计算机与人工智能学院);Engineering Research Center of Intelligent Swarm Systems, Ministry of Education(教育部智能群体系统工程研究中心);National Supercomputing Center In Zhengzhou(郑州国家超级计算中心);Zhejiang University(浙江大学)

Dong Chen, Shilin Zhang, Fei Gao, Yueting Zhuang, Siliang Tang, Qidong Liu, Mingliang Xu

AI总结:

本文提出逻辑蒸馏(LD)框架,通过将大模型的复杂指令实例化为函数库并微调小模型按函数学习规划与决策逻辑,使小模型在决策任务中达到或超越大模型性能。

摘要:

大语言模型(LLMs)因其强大的逻辑推理能力而受到越来越多的关注。通常,需要付费接口的较大规模大语言模型(L-LLMs)在性能上显著优于可部署在多种设备上的较小规模大语言模型(S-LLMs)。知识蒸馏(KD)旨在赋予S-LLMs以L-LLMs的能力,但S-LLMs仅模仿L-LLMs的输出,无法获得强大的逻辑推理能力。因此,在面对需要逻辑推理能力的规划与决策任务时,S-LLMs无能为力。为解决上述挑战,我们提出了一种名为逻辑蒸馏(LD)的新框架。首先,LD利用L-LLMs将复杂指令实例化为离散函数,并说明其用法以建立函数库。随后,基于该函数库,LD对S-LLMs进行微调,使其学习L-LLMs在规划与决策中所使用的逻辑。在测试阶段,LD使用检索器根据指令和当前状态识别最相关的top-$K$个函数,这些函数将被S-LLMs选择并调用。最终,S-LLMs按函数逐段生成规划与决策结果。相关实验表明,在LD的辅助下,S-LLMs能够在规划与决策任务中取得优异结果,其表现可与L-LLMs相当,甚至超越L-LLMs。

英文摘要:

Large language models (LLMs) have garnered increasing attention owing to their powerful logical reasoning capabilities. Generally, larger LLMs (L-LLMs) that require paid interfaces exhibit significantly superior performance compared to smaller LLMs (S-LLMs) that can be deployed on a variety of devices. Knowledge distillation (KD) aims to empower S-LLMs with the capabilities of L-LLMs, while S-LLMs merely mimic the outputs of L-LLMs, failing to get the powerful logical reasoning capabilities. Consequently, S-LLMs are helpless when it comes to planning and decision-making tasks that require logical reasoning capabilities. To tackle the identified challenges, we propose a novel framework called Logic Distillation (LD). Initially, LD employs L-LLMs to instantiate complex instructions into discrete functions and illustrates their usage to establish a function base. Subsequently, based on the function base, LD fine-tunes S-LLMs to learn the logic employed by L-LLMs in planning and decision-making. During testing, LD utilizes a retriever to identify the top-$K$ relevant functions based on instructions and current states, which will be selected and invoked by S-LLMs. Ultimately, S-LLMs yield planning and decision-making outcomes, function by function. Relevant experiments demonstrate that with the assistance of LD, S-LLMs can achieve outstanding results in planning and decision-making tasks, comparable to, or even surpassing, those of L-LLMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑