arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2411.15399cs.PFcs.DCcs.LG
收藏

少即是多:优化边缘设备上LLM执行的函数调用

Less is More: Optimizing Function Calling for LLM Execution on Edge Devices

School of Electrical, Computer and Biomedical Engineering, Southern Illinois University(南伊利诺伊大学电气、计算机与生物医学工程学院);The University of Texas at Austin(德克萨斯大学奥斯汀分校)

Varatheepan Paramanayakam, Andreas Karatzas, Iraklis Anagnostopoulos, Dimitrios Stamoulis

AI总结:

本文提出无需微调的Less-is-More函数调用方案,通过动态减少LLM可用工具数量,在边缘设备上提升智能体成功率,并将执行时间最多降低70%、功耗最多降低40%。

摘要:

基础模型先进的函数调用能力为部署智能体执行复杂API任务开辟了新的可能性。然而,管理大量数据并与众多API交互使得函数调用硬件密集且成本高昂,尤其是在边缘设备上。当前的大语言模型(LLMs)在边缘端进行函数调用时面临困难,因为它们无法处理复杂输入或有效管理多个工具。这导致任务完成准确率低、延迟增加和功耗升高。在这项工作中,我们提出了Less-is-More,一种无需微调的新型函数调用方案,用于动态工具选择。我们的方法基于一个关键洞察:有选择地减少LLMs可用的工具数量能显著提升其在边缘设备上的函数调用性能、执行时间和功耗效率。在边缘硬件上使用最先进的LLMs进行的实验结果表明,智能体成功率有所提升,执行时间最多减少70%,功耗最多降低40%。

英文摘要:

The advanced function-calling capabilities of foundation models open up new possibilities for deploying agents to perform complex API tasks. However, managing large amounts of data and interacting with numerous APIs makes function calling hardware-intensive and costly, especially on edge devices. Current Large Language Models (LLMs) struggle with function calling at the edge because they cannot handle complex inputs or manage multiple tools effectively. This results in low task-completion accuracy, increased delays, and higher power consumption. In this work, we introduce Less-is-More, a novel fine-tuning-free function-calling scheme for dynamic tool selection. Our approach is based on the key insight that selectively reducing the number of tools available to LLMs significantly improves their function-calling performance, execution time, and power efficiency on edge devices. Experimental results with state-of-the-art LLMs on edge hardware show agentic success rate improvements, with execution time reduced by up to 70% and power consumption by up to 40%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑