arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.04253cs.CL
收藏

AnyTool:用于大规模API调用的自反思分层智能体

AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Calls

Tsinghua University(清华大学);Microsoft Research Asia(微软亚洲研究院);University of Waterloo(滑铁卢大学)

Yu Du, Fangyun Wei, Hongyang Zhang

AI总结:

AnyTool是一个利用GPT-4函数调用、结合分层API检索、求解器和自反思机制的大型语言模型智能体,通过修订评估协议并引入AnyToolBench,在ToolBench上平均通过率比ToolLLM高35.4%。

摘要:

我们引入了AnyTool,这是一个旨在彻底改变利用大量工具解决用户查询方式的大型语言模型智能体。我们使用了来自Rapid API的超过16,000个API,并假设这些API中的一部分可能解决查询。AnyTool主要包含三个要素:一个具有分层结构的API检索器,一个旨在使用选定的一组API候选来解决用户查询的求解器,以及一个自反思机制,如果初始解决方案被证明不可行,该机制会重新激活AnyTool。AnyTool由GPT-4的函数调用功能驱动,无需训练外部模块。我们还重新审视了先前工作引入的评估协议,并识别出该协议中的一个局限性,该局限性导致人为的高通过率。通过修订评估协议以更好地反映实际应用场景,我们引入了一个额外的基准,称为AnyToolBench。跨多个数据集的实验表明,我们的AnyTool优于强基线,如ToolLLM和专为工具利用定制的GPT-4变体。例如,在ToolBench上,AnyTool在平均通过率方面比ToolLLM高出+35.4%。代码将在https://github.com/dyabel/AnyTool提供。

英文摘要:

We introduce AnyTool, a large language model agent designed to revolutionize the utilization of a vast array of tools in addressing user queries. We utilize over 16,000 APIs from Rapid API, operating under the assumption that a subset of these APIs could potentially resolve the queries. AnyTool primarily incorporates three elements: an API retriever with a hierarchical structure, a solver aimed at resolving user queries using a selected set of API candidates, and a self-reflection mechanism, which re-activates AnyTool if the initial solution proves impracticable. AnyTool is powered by the function calling feature of GPT-4, eliminating the need for training external modules. We also revisit the evaluation protocol introduced by previous works and identify a limitation in this protocol that leads to an artificially high pass rate. By revising the evaluation protocol to better reflect practical application scenarios, we introduce an additional benchmark, termed AnyToolBench. Experiments across various datasets demonstrate the superiority of our AnyTool over strong baselines such as ToolLLM and a GPT-4 variant tailored for tool utilization. For instance, AnyTool outperforms ToolLLM by +35.4% in terms of average pass rate on ToolBench. Code will be available at https://github.com/dyabel/AnyTool.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑