arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.02141cs.LGcs.AIcs.CL
收藏

向量空间中工具表示的高效且可扩展估计

Efficient and Scalable Estimation of Tool Representations in Vector Space

UC Berkeley(加州大学伯克利分校);POSCO HOLDINGS(浦项控股);LBNL(劳伦斯伯克利国家实验室)

Suhong Moon, Siddharth Jha, Lutfi Eren Erdogan, Sehoon Kim, Woosang Lim, Kurt Keutzer, Amir Gholami

AI总结:

本文提出Tool2Vec、ToolRefiner和MLC三种工具检索方法,并构建ToolBank数据集,通过小型编码器模型实现高效可扩展的工具检索,在ToolBench和ToolBank上显著提升Recall@K。

摘要:

近年来,函数调用和工具使用方面的进展显著增强了大语言模型(LLMs)的能力,使其能够与外部信息源交互并执行复杂任务。然而,当可用工具数量庞大时,LLMs有限的上下文窗口带来了挑战,因此需要高效的方法来管理提示长度并保持准确性。现有方法,如对LLMs进行微调或利用其推理能力,要么需要频繁重新训练,要么会产生显著的延迟开销。一种更高效的解决方案是训练较小的模型来为给定查询检索最相关的工具,但这需要高质量、领域特定的数据。为了应对这些挑战,我们提出了一个用于工具检索应用合成数据生成的新框架,以及一种使用小型编码器模型的高效数据驱动工具检索策略。借助LLMs的能力,我们创建了ToolBank,这是一个反映真实人类用户使用情况的新工具检索数据集。在工具检索方法上,我们提出了新颖的方法:(1)Tool2Vec:用于工具检索的使用驱动的工具嵌入生成;(2)ToolRefiner:一种分阶段检索方法,迭代地提高检索到的工具的质量;(3)MLC:将工具检索构建为多标签分类问题。凭借这些新方法,我们在ToolBench数据集上的Recall@K提升了最高27.28,在ToolBank上的Recall@K提升了30.5。此外,我们还提供了进一步的实验结果,以严格验证我们的方法。我们的代码可在\url{https://github.com/SqueezeAILab/Tool2Vec}获取。

英文摘要:

Recent advancements in function calling and tool use have significantly enhanced the capabilities of large language models (LLMs) by enabling them to interact with external information sources and execute complex tasks. However, the limited context window of LLMs presents challenges when a large number of tools are available, necessitating efficient methods to manage prompt length and maintain accuracy. Existing approaches, such as fine-tuning LLMs or leveraging their reasoning capabilities, either require frequent retraining or incur significant latency overhead. A more efficient solution involves training smaller models to retrieve the most relevant tools for a given query, although this requires high quality, domain-specific data. To address those challenges, we present a novel framework for generating synthetic data for tool retrieval applications and an efficient data-driven tool retrieval strategy using small encoder models. Empowered by LLMs, we create ToolBank, a new tool retrieval dataset that reflects real human user usages. For tool retrieval methodologies, we propose novel approaches: (1) Tool2Vec: usage-driven tool embedding generation for tool retrieval, (2) ToolRefiner: a staged retrieval method that iteratively improves the quality of retrieved tools, and (3) MLC: framing tool retrieval as a multi-label classification problem. With these new methods, we achieve improvements of up to 27.28 in Recall@K on the ToolBench dataset and 30.5 in Recall@K on ToolBank. Additionally, we present further experimental results to rigorously validate our methods. Our code is available at \url{https://github.com/SqueezeAILab/Tool2Vec}

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑