arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.01108cs.CLcs.LG
收藏

多智能体系统中的推理能力:局限、挑战与以人为中心的解决方案

Reasoning Capacity in Multi-Agent Systems: Limitations, Challenges and Human-Centered Solutions

Megagon Labs(美格根实验室);Carnegie Mellon University(卡内基梅隆大学)

Pouya Pezeshkpour, Eser Kandogan, Nikita Bhutani, Sajjadur Rahman, Tom Mitchell, Estevam Hruschka

AI总结:

本文提出推理能力作为统一标准,用于在多智能体系统中整合约束并连接组件,通过自我反思和人类反馈解决推理局限,实现更全面的评估。

摘要:

大型语言模型(LLMs)在各种任务中的卓越表现,为在生产环境中利用它们带来了许多机遇和挑战。为了实际采用LLMs,多智能体系统在利用现有专有数据及模型处理复杂现实任务的企业平台的大背景下,有望增强、集成和编排LLMs。尽管这些系统取得了巨大成功,但当前的方法依赖于狭窄、单一目标的优化和评估目标,往往忽视了现实场景中的潜在约束,包括受限的预算、资源和时间。此外,解释、分析和调试这些系统需要各组件相互关联地进行评估。这一需求在现有方法论下目前无法实现。在这篇立场论文中,我们引入了推理能力的概念,作为统一标准,以便在优化过程中整合约束,并建立系统内不同组件之间的联系,这也使得评估方法更加全面和综合。我们给出了推理能力的正式定义,并阐述了其在识别系统各组件局限中的实用性。然后,我们论证了如何通过自我反思过程来解决这些局限,其中利用人类反馈来缓解推理中的不足,并增强系统的整体一致性。

英文摘要:

Remarkable performance of large language models (LLMs) in a variety of tasks brings forth many opportunities as well as challenges of utilizing them in production settings. Towards practical adoption of LLMs, multi-agent systems hold great promise to augment, integrate, and orchestrate LLMs in the larger context of enterprise platforms that use existing proprietary data and models to tackle complex real-world tasks. Despite the tremendous success of these systems, current approaches rely on narrow, single-focus objectives for optimization and evaluation, often overlooking potential constraints in real-world scenarios, including restricted budgets, resources and time. Furthermore, interpreting, analyzing, and debugging these systems requires different components to be evaluated in relation to one another. This demand is currently not feasible with existing methodologies. In this postion paper, we introduce the concept of reasoning capacity as a unifying criterion to enable integration of constraints during optimization and establish connections among different components within the system, which also enable a more holistic and comprehensive approach to evaluation. We present a formal definition of reasoning capacity and illustrate its utility in identifying limitations within each component of the system. We then argue how these limitations can be addressed with a self-reflective process wherein human-feedback is used to alleviate shortcomings in reasoning and enhance overall consistency of the system.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑