arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.00608cs.CLcs.LG
收藏

TinyAgent:在边缘端进行函数调用

TinyAgent: Function Calling at the Edge

UC Berkeley(加州大学伯克利分校);ICSI(国际计算机科学研究所)

Lutfi Eren Erdogan, Nicholas Lee, Siddharth Jha, Sehoon Kim, Ryan Tabrizi, Suhong Moon, Coleman Hooper, Gopala Anumanchipalli, Kurt Keutzer, Amir Gholami

AI总结:

本文提出 TinyAgent 端到端框架,通过高质量函数调用数据微调 1.1B/7B 小模型,并结合工具检索与量化,在 MacBook 边缘端实现可媲美甚至超越 GPT-4-Turbo 的本地函数调用智能体。

摘要:

近期的大型语言模型(LLM)推动了先进智能体系统的发展,这些系统能够集成各种工具和 API,通过函数调用来完成用户查询。然而,由于这些 LLM 模型规模庞大且计算需求高,通常需要基于云的基础设施,因此尚未有人探索在边缘端部署它们。为此,我们提出了 TinyAgent,这是一个端到端框架,用于训练和部署能够进行函数调用的任务专用小型语言模型智能体,从而在边缘端驱动智能体系统。我们首先展示如何通过 LLMCompiler 框架为开源模型实现准确的函数调用。随后,我们系统地构建了一个高质量函数调用数据集,并用其微调两个小型语言模型:TinyAgent-1.1B 和 7B。为实现高效推理,我们提出了一种新颖的工具检索方法以缩短输入提示长度,并利用量化进一步加快推理速度。作为驱动应用,我们展示了一个面向 Apple MacBook 的本地类 Siri 系统,该系统可通过文本或语音输入执行用户命令。结果表明,我们的模型在完全部署于边缘端的同时,能够达到甚至超越 GPT-4-Turbo 等更大模型的函数调用能力。我们开源了数据集、模型和可安装软件包,并提供了 MacBook 助手智能体的演示视频。

英文摘要:

Recent large language models (LLMs) have enabled the development of advanced agentic systems that can integrate various tools and APIs to fulfill user queries through function calling. However, the deployment of these LLMs on the edge has not been explored since they typically require cloud-based infrastructure due to their substantial model size and computational demands. To this end, we present TinyAgent, an end-to-end framework for training and deploying task-specific small language model agents capable of function calling for driving agentic systems at the edge. We first show how to enable accurate function calling for open-source models via the LLMCompiler framework. We then systematically curate a high-quality dataset for function calling, which we use to fine-tune two small language models, TinyAgent-1.1B and 7B. For efficient inference, we introduce a novel tool retrieval method to reduce the input prompt length and utilize quantization to further accelerate the inference speed. As a driving application, we demonstrate a local Siri-like system for Apple's MacBook that can execute user commands through text or voice input. Our results show that our models can achieve, and even surpass, the function-calling capabilities of larger models like GPT-4-Turbo, while being fully deployed at the edge. We open-source our dataset, models, and installable package and provide a demo video for our MacBook assistant agent.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑