HyFunc: 通过混合模型级联和动态模板加速基于LLM的功能调用以实现代理AI
HyFunc: Accelerating LLM-based Function Calls for Agentic AI through Hybrid-Model Cascade and Dynamic Templating
AI总结:
HyFunc通过混合模型级联和动态模板技术,高效解决代理AI中功能调用的冗余问题,实现低延迟和高性能。
摘要:
尽管代理AI系统依赖LLM将用户意图转换为结构化功能调用,但这一过程存在计算冗余,导致高推理延迟,阻碍了实时应用。本文识别并解决了三个关键冗余问题:(1)为每个请求处理大量功能描述的冗余;(2)使用大型、缓慢模型生成整个、往往可预测的标记序列的冗余;(3)生成固定、模板化参数语法的冗余。我们引入了HyFunc,一种新型框架,系统地消除这些低效。HyFunc采用混合模型级联,其中大型模型将用户意图提炼为一个“软标记”。该标记引导轻量检索器选择相关功能,并指导较小的前缀微调模型生成最终调用,从而避免大型模型的冗余上下文处理和完整序列生成。为了消除语法冗余,我们的“动态模板”技术在扩展的vLLM引擎中即时注入模板化参数语法。为了避免可能的泛化限制,我们在未见过的基准数据集BFCL上评估HyFunc。实验结果表明,HyFunc在效率和性能之间实现了良好的平衡。它达到了0.828秒的推理延迟,优于所有基线模型,并达到了80.1%的性能,超过了参数规模相当的所有模型。这些结果表明,HyFunc为代理AI提供了一种更高效的范式。我们的代码在https://github.com/MrBlankness/HyFunc上公开可用。
英文摘要:
While agentic AI systems rely on LLMs to translate user intent into structured function calls, this process is fraught with computational redundancy, leading to high inference latency that hinders real-time applications. This paper identifies and addresses three key redundancies: (1) the redundant processing of a large library of function descriptions for every request; (2) the redundant use of a large, slow model to generate an entire, often predictable, token sequence; and (3) the redundant generation of fixed, boilerplate parameter syntax. We introduce HyFunc, a novel framework that systematically eliminates these inefficiencies. HyFunc employs a hybrid-model cascade where a large model distills user intent into a single "soft token." This token guides a lightweight retriever to select relevant functions and directs a smaller, prefix-tuned model to generate the final call, thus avoiding redundant context processing and full-sequence generation by the large model. To eliminate syntactic redundancy, our "dynamic templating" technique injects boilerplate parameter syntax on-the-fly within an extended vLLM engine. To avoid potential limitations in generalization, we evaluate HyFunc on an unseen benchmark dataset, BFCL. Experimental results demonstrate that HyFunc achieves an excellent balance between efficiency and performance. It achieves an inference latency of 0.828 seconds, outperforming all baseline models, and reaches a performance of 80.1%, surpassing all models with a comparable parameter scale. These results suggest that HyFunc offers a more efficient paradigm for agentic AI. Our code is publicly available at https://github.com/MrBlankness/HyFunc.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。