arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2502.00510cs.AIcs.CL
收藏

通过 Shapley value 理解和优化智能体工作流

Understanding and Optimizing Agentic Workflows via Shapley value

Yingxuan Yang, Bo Huang, Siyuan Qi, Chao Feng, Haoyi Hu, Yuxuan Zhu, Jinbo Hu, Haoran Zhao, Ziyi He, Xiao Liu, Muning Wen, Zongyu Wang, Lin Qiu, Xuezhi Cao, Xun… 展开作者

Yingxuan Yang, Bo Huang, Siyuan Qi, Chao Feng, Haoyi Hu, Yuxuan Zhu, Jinbo Hu, Haoran Zhao, Ziyi He, Xiao Liu, Muning Wen, Zongyu Wang, Lin Qiu, Xuezhi Cao, Xunliang Cai, Yong Yu, Weinan Zhang

AI总结:

本文提出 ShapleyFlow 框架,首次利用合作博弈论中的 Shapley value 对智能体工作流进行细粒度贡献归因,通过评估所有组件配置发现任务特定的最优工作流,并在 7 个场景、超 1,500 个任务上验证其优于单一 LLM 方案。

摘要:

智能体工作流已成为构建复杂 AI 系统的主导范式,通过编排规划、推理、动作执行和反思等专用组件来处理复杂的现实任务。然而,由于复杂的组件间相互依赖关系以及缺乏有原则的归因方法,系统性地分析和优化这些工作流仍具挑战性。在本研究中,我们提出了 ShapleyFlow,这是首个采用合作博弈论来分析和优化智能体工作流的框架。通过应用 Shapley value 评估所有可能的组件配置,ShapleyFlow 能够实现对每个组件贡献的细粒度归因,并促进识别任务特定的最优配置。通过在导航、数学和 OS 等 7 个场景中评估的构建数据集,我们展示了 3 项关键贡献:(1) 理论框架:一种用于智能体工作流中贡献归因的有原则的博弈论方法。(2) 最优工作流发现:ShapleyFlow 识别出任务特定的组件配置,在所有测试任务中始终优于依赖单一 LLM 的工作流。(3) 综合分析:我们构建并分析了超过 1,500 个任务,为跨多个领域优化工作流提供了可操作的见解和设计指南。

英文摘要:

Agentic workflows have become the dominant paradigm for building complex AI systems, orchestrating specialized components, such as planning, reasoning, action execution, and reflection, to tackle sophisticated real-world tasks. However, systematically analyzing and optimizing these workflows remains challenging due to intricate component interdependencies and the lack of principled attribution methods. In this work, we introduce ShapleyFlow, the first framework that employs cooperative game theory to analyze and optimize agentic workflows. By applying the Shapley value to evaluate all possible component configurations, ShapleyFlow enables fine-grained attribution of each component's contribution and facilitates the identification of task-specific optimal configurations. Through a constructed dataset evaluated across 7 scenarios, such as navigation, math and OS, we demonstrate 3 key contributions: (1) Theoretical Framework: a principled game-theoretic approach for the attribution of contributions in agentic workflows. (2) Optimal Workflow Discovery: ShapleyFlow identifies task-specific component configurations that consistently outperform workflows relying on a single LLM across all tested tasks. (3) Comprehensive Analysis: we construct and analyze over 1,500 tasks, providing actionable insights and design guidelines for optimizing workflows across multiple domains.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑