arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.21784cs.AIcs.CLcs.LGcs.MA
收藏

MARCO:多智能体实时聊天编排

MARCO: Multi-Agent Real-time Chat Orchestration

Retail Business Services, Amazon(亚马逊零售商业服务);AWS Bedrock, Amazon(亚马逊AWS Bedrock)

Anubhav Shrimal, Stanley Kanagaraj, Kriti Biswas, Swarnalatha Raghuraman, Anish Nediyanchath, Yi Zhang, Promod Yenigalla

AI总结:

本研究提出多智能体实时聊天编排框架MARCO,通过防护机制解决大语言模型执行复杂多步任务的错误问题,在两类对话数据集上实现高准确率,同时显著降低延迟与成本,支持跨领域任务适配。

摘要:

大语言模型的发展推动了多智能体框架的开发,以解决复杂的现实问题,例如自动化那些需要与各类工具交互、推理及人机协作的任务。我们提出了MARCO——一种基于大语言模型(LLM)实现任务自动化的多智能体实时聊天编排框架。MARCO解决了利用LLM执行复杂多步骤任务时的核心挑战。它内置了稳健的防护机制,用于引导LLM行为、验证输出,并从输出格式不一致、函数与参数幻觉、领域知识缺失导致的错误中恢复。通过大量实验,我们证明了MARCO的卓越性能:在数字餐饮服务平台对话数据集和零售对话数据集上的任务执行准确率分别达到94.48%和92.74%,同时延迟降低44.91%,成本减少33.71%。我们还报告了防护机制对性能提升的作用,并对比了多种开源和专有LLM模型的表现。MARCO的模块化通用设计使其能够适配跨领域的任务自动化,并通过多轮交互执行复杂用例。

英文摘要:

Large language model advancements have enabled the development of multi-agent frameworks to tackle complex, real-world problems such as to automate tasks that require interactions with diverse tools, reasoning, and human collaboration. We present MARCO, a Multi-Agent Real-time Chat Orchestration framework for automating tasks using LLMs. MARCO addresses key challenges in utilizing LLMs for complex, multi-step task execution. It incorporates robust guardrails to steer LLM behavior, validate outputs, and recover from errors that stem from inconsistent output formatting, function and parameter hallucination, and lack of domain knowledge. Through extensive experiments we demonstrate MARCO's superior performance with 94.48% and 92.74% accuracy on task execution for Digital Restaurant Service Platform conversations and Retail conversations datasets respectively along with 44.91% improved latency and 33.71% cost reduction. We also report effects of guardrails in performance gain along with comparisons of various LLM models, both open-source and proprietary. The modular and generic design of MARCO allows it to be adapted for automating tasks across domains and to execute complex usecases through multi-turn interactions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑