专业化角色，混合部署：推动LLM代理团队的成本-精度前沿
Specialize Roles, Mix Deployments: Pushing the Cost-Accuracy Frontier of LLM Agent Teams
AI总结：
提出AgentCARD基准，通过角色感知评估和帕累托前沿分析，发现异构团队在成本-精度前沿上优于同构团队，混合部署可降低12倍成本。
摘要：
LLM代理越来越多地被部署为多角色团队，其中任务被分配到专业角色，如规划者、执行者和验证者。在这些系统中，成本和精度不再是单一模型的属性：它们取决于每个角色由哪个模型填充以及模型部署在哪里，包括API、自托管和混合部署。现有的代理基准通常评估固定模型或固定代理配置，因此对成本-精度最优部署提供的指导有限。我们引入了AgentCARD，一个角色感知的基准套件，用于评估跨角色分配和部署模式的LLM代理团队。AgentCARD结合了角色分解的评估框架、统一的API/自托管成本模型、帕累托前沿分析以及基于Shapley的诊断方法，用于识别角色瓶颈。我们的评估表明，异构团队始终占据成本-精度前沿。与成本等价的同构团队相比，它们将精度提高了高达44%，或者通过混合部署以高达12倍更低的每任务成本匹配最强的同构团队。我们进一步发现，最佳角色分配是领域依赖的：一些领域是规划者瓶颈，而其他领域是执行者瓶颈。最后，AgentCARD扩展到超越规划者-执行者团队的工作流程，包括如验证等额外角色，并支持随着新领域和团队结构的出现进行持续评估。我们的代码发布在：https://this https URL
英文摘要：
LLM agents are increasingly deployed as multi-role teams, where tasks are divided across specialized roles such as planner, executor, and verifier. In these systems, cost and accuracy are no longer properties of a single model: they depend on which model fills each role and where it is hosted, including API, self-hosted, and hybrid deployment. Existing agentic benchmarks typically evaluate fixed models or fixed agent configurations, and therefore offer limited guidance for cost-accuracy-optimal deployment. We introduce AgentCARD, a role-aware benchmark suite for evaluating LLM agent teams across role assignment and deployment mode. AgentCARD combines a role-decomposed evaluation harness, a unified API/self-hosted cost model, Pareto-frontier analysis, and a Shapley-based diagnostic for identifying role bottlenecks. Our evaluation shows that heterogeneous teams consistently occupy the cost-accuracy frontier. They improve accuracy by up to $44\%$ over cost-equivalent homogeneous teams, or match the strongest homogeneous team at up to $12\times$ lower per-task cost through hybrid deployment. We further find that the best role assignment is domain-dependent: some domains are planner-bottlenecked, while others are executor-bottlenecked. Finally, AgentCARD extends beyond planner--executor teams to workflows with additional roles such as verification, and supports continual evaluation as new domains and team structures emerge. Our code is released at: https://github.com/Auto-CAP/AgentCAP
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
