章鱼v2:用于超级代理的设备端语言模型
Octopus v2: On-device language model for super agent
AI总结:
章鱼v2通过设备端20亿参数模型,在准确性和延迟上超越GPT-4,同时将上下文长度减少95%,提升边缘设备应用性能。
摘要:
语言模型在各种软件应用中已显示出有效性,特别是在与自动工作流相关的任务中。这些模型具备调用函数的关键能力,这对于创建AI代理至关重要。尽管在云环境中大规模语言模型表现出色,但它们通常与隐私和成本问题相关。当前用于函数调用的设备端模型面临延迟和准确性的问题。我们的研究提出了一种新方法,使具有20亿参数的设备端模型在准确性和延迟方面超越GPT-4,并将上下文长度减少95%。与具有基于RAG的函数调用机制的Llama-7B相比,我们的方法将延迟降低了35倍。该方法将延迟降低到适合在生产环境中部署在各种边缘设备上的水平,符合实际应用的性能要求。
英文摘要:
Language models have shown effectiveness in a variety of software applications, particularly in tasks related to automatic workflow. These models possess the crucial ability to call functions, which is essential in creating AI agents. Despite the high performance of large-scale language models in cloud environments, they are often associated with concerns over privacy and cost. Current on-device models for function calling face issues with latency and accuracy. Our research presents a new method that empowers an on-device model with 2 billion parameters to surpass the performance of GPT-4 in both accuracy and latency, and decrease the context length by 95\%. When compared to Llama-7B with a RAG-based function calling mechanism, our method enhances latency by 35-fold. This method reduces the latency to levels deemed suitable for deployment across a variety of edge devices in production environments, aligning with the performance requisites for real-world applications.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。