arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.18890cs.AI
收藏

改进小规模大型语言模型在推理任务中的函数调用能力

Improving Small-Scale Large Language Models Function Calling for Reasoning Tasks

Università di Pisa(比萨大学);Dipartimento di Ingegneria dell’Informazione(信息工程系)

Graziano A. Manduzio, Federico A. Galatolo, Mario G. C. A. Cimino, Enzo Pasquale Scilingo, Lorenzo Cominelli

AI总结:

本文提出用大规模LLM生成正确与错误函数调用推理链偏好数据,并通过RLHF中的DPO训练小规模模型,以较低计算成本提升其数学和逻辑推理任务的函数调用能力。

摘要:

近期大型语言模型(LLM)的进展展现出在自然语言理解与生成方面的卓越能力。尽管这些模型在通用复杂推理任务中表现出色,但它们在数学问题求解和逻辑推理方面仍面临挑战。为解决这些局限,研究人员探索了函数调用能力,使LLM能够执行所提供的函数,并利用其输出完成任务。然而,由于大规模LLM在训练和推理阶段需要耗费昂贵的计算资源,专注于特定任务而使用大规模LLM可能效率很低。本研究提出了一个新颖框架,用于训练较小的语言模型进行函数调用,重点面向特定的逻辑与数学推理任务。该方法旨在利用函数调用提升小规模模型在这些任务上的表现,同时确保较高的准确率。我们的框架采用一个智能体:在给定问题和一组可调用函数的情况下,通过将可用函数的描述和示例注入提示词来查询LLM,并在逐步推理链中管理这些函数调用。该过程用于根据一个大规模LLM创建包含正确与错误推理链聊天补全的数据集。随后,使用该数据集,通过基于人类反馈的强化学习(RLHF)训练较小的LLM,具体采用直接偏好优化(DPO)技术。实验结果表明,所提出方法如何在模型规模与性能之间平衡权衡,从而提升较小模型在推理任务中的函数调用能力。

英文摘要:

Recent advancements in Large Language Models (LLMs) have demonstrated exceptional capabilities in natural language understanding and generation. While these models excel in general complex reasoning tasks, they still face challenges in mathematical problem-solving and logical reasoning. To address these limitations, researchers have explored function calling abilities, allowing LLMs to execute provided functions and utilize their outputs for task completion. However, concentrating on specific tasks can be very inefficient for large-scale LLMs to be used, because of the expensive cost of training and inference stages they need in terms of computational resources. This study introduces a novel framework for training smaller language models in function calling, focusing on specific logical and mathematical reasoning tasks. The approach aims to improve performances of small-scale models for these tasks using function calling, ensuring a high level of accuracy. Our framework employs an agent that, given a problem and a set of callable functions, queries the LLM by injecting a description and examples of the usable functions into the prompt and managing their calls in a step-by-step reasoning chain. This process is used to create a dataset of correct and incorrect reasoning chain chat completions from a large-scale LLM. This dataset is used to train a smaller LLM using Reinforcement Learning from Human Feedback (RLHF), specifically employing the Direct Preference Optimization (DPO) technique. Experimental results demonstrate how the proposed approach balances the trade-off between model size and performance, improving the ability of function calling for reasoning tasks, in smaller models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑