在不完美选择与计算成本下设计智能体AI工作流组合

中文摘要 AI 辅助

智能体AI系统通常通过多种工作流来处理同一任务，这些工作流在推理策略、验证结构和计算成本上有所不同。一种自然的部署策略是使用平均性能最高的工作流，但这可能不是最优的，因为不同的工作流可能在不同的实例上取得成功。我们研究了一种组合与选择器范式，其中企业运行多个工作流执行，并在观察其输出后选择最终答案。额外的执行可能会发现最佳独立工作流遗漏的正确答案，但它们消耗计算资源并引入可能干扰最终选择的似是而非的干扰项。我们将此表述为一个工作流组合问题，其中企业联合选择运行规模和跨工作流类型的分配。我们通过赔率提升指数总结选择器质量，并推导出工作流多样性的价值的严格界限。对于有限的工作流池，我们开发了精确公式、线性规划松弛、随机舍入程序和可计算性能证书。对于大规模隐式工作流类别，我们推导出有限维对偶和使用定价预言机的椭球方法，以识别具有高加权准确率净经常性计算成本的工作流。在弱条件下，该方法通过多项式次预言机调用获得松弛的近似最优解。我们在三个数据集上评估该框架：ABCD、Schema-Guided Dialogue和HotpotQA。相对于最佳独立工作流，组合优化分别将保留选择器准确率提高了3.1、7.5和0.9个百分点。对偶引导的工作流生成在ABCD上增加了3.5个百分点，在HotpotQA上增加了24.1个百分点，在Schema-Guided Dialogue上没有额外增益。

英文摘要

Agentic AI systems often approach the same task through multiple workflows that differ in reasoning strategy, verification structure, and compute cost. A natural deployment policy is to use the workflow with the highest average performance, but this can be suboptimal because different workflows may succeed on different instances. We study a portfolio-and-selector paradigm in which a firm runs multiple workflow executions and selects the final answer after observing their outputs. Additional executions may uncover correct answers that the best standalone workflow misses, but they consume compute and introduce plausible distractors that complicate final selection. We formulate this as a workflow portfolio problem in which the firm jointly chooses run size and allocation across workflow types. We summarize selector quality through an odds-lift index and derive sharp bounds on the value of workflow variety. For finite workflow pools, we develop exact formulations, linear programming relaxations, randomized rounding procedures, and computable performance certificates. For large implicit workflow classes, we derive a finite-dimensional dual and an ellipsoid method using a pricing oracle to identify workflows with high weighted accuracy net of recurring compute cost. Under a weak condition, the method obtains a near-optimal solution to the relaxation with polynomially many oracle calls. We evaluate the framework on three datasets: ABCD, Schema-Guided Dialogue, and HotpotQA. Relative to the best standalone workflow, portfolio optimization improves held-out selector accuracy by 3.1, 7.5, and 0.9 percentage points, respectively. Dual-guided workflow generation adds 3.5 points on ABCD and 24.1 on HotpotQA, with no additional gain on Schema-Guided Dialogue.

发表机构

• Ross School of Business, University of Michigan（密歇根大学罗斯商学院）
• TrueFoundry
• School of Management, University of San Francisco（旧金山大学管理学院）

机构由 AI 辅助整理，请以论文原文为准。