arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2411.04535cs.CLcs.AI
收藏

元推理改进大语言模型中的工具使用

Meta-Reasoning Improves Tool Use in Large Language Models

Imperial College London(伦敦帝国学院)

Lisa Alazraki, Marek Rei

AI总结:

本文提出TECTON两阶段系统,通过先推理生成候选工具再进行元推理选择最终工具,显著提升大语言模型在数学推理任务中的工具使用效果。

摘要:

外部工具帮助大语言模型在原本通常会失败的任务上取得成功。在现有框架中,测试时选择工具依赖于朴素的贪心解码,无论模型是在工具标注数据上微调过,还是通过上下文示例进行提示。相比之下,我们发现收集并在一组合适的候选工具中进行选择,更有可能导向最优选择。我们提出通过元推理进行工具选择(TECTON),这是一个两阶段系统,首先对任务进行推理,并使用自定义微调的语言建模头输出候选工具。然后,在禁用自定义头的情况下,它进行元推理(即对先前的推理过程进行推理)以做出最终选择。我们表明,TECTON在一系列数学推理数据集上带来了显著提升——无论是在分布内还是分布外。

英文摘要:

External tools help large language models succeed at tasks where they would otherwise typically fail. In existing frameworks, choosing tools at test time relies on naive greedy decoding, regardless of whether the model has been fine-tuned on tool-annotated data or prompted with in-context examples. In contrast, we find that gathering and choosing among a suitable set of candidate tools has greater potential to lead to an optimal selection. We present Tool selECTion via meta-reasONing (TECTON), a two-phase system that first reasons over a task and outputs candidate tools using a custom fine-tuned language modelling head. Then, with the custom head disabled, it meta-reasons (i.e., it reasons over the previous reasoning process) to make a final choice. We show that TECTON results in substantial gains--both in-distribution and out-of-distribution--on a range of math reasoning datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑