构建、评判与优化：多智能体消费者助手持续改进的蓝图
Build, Judge, Optimize: A Blueprint for Continuous Improvement of Multi-Agent Consumer Assistants
AI总结：
本文提出评估和优化对话购物助手的实用蓝图，通过生产级AI杂货助手示例，介绍多维评估标准和校准的LLM评判流程，并提出两种互补的提示优化策略。
摘要：
对话购物助手（CSAs）是代理AI的有吸引力应用，但从原型到生产面临两大挑战：如何评估多轮交互和如何优化紧密耦合的多智能体系统。杂货购物进一步加剧了这些困难，因为用户请求通常不明确、高度偏好敏感，并受预算和库存等因素限制。本文提出评估和优化对话购物助手的实用蓝图，通过生产级AI杂货助手示例，介绍多维评估标准和校准的LLM评判流程，并提出两种互补的提示优化策略。
英文摘要：
Conversational shopping assistants (CSAs) represent a compelling application of agentic AI, but moving from prototype to production reveals two underexplored challenges: how to evaluate multi-turn interactions and how to optimize tightly coupled multi-agent systems. Grocery shopping further amplifies these difficulties, as user requests are often underspecified, highly preference-sensitive, and constrained by factors such as budget and inventory. In this paper, we present a practical blueprint for evaluating and optimizing conversational shopping assistants, illustrated through a production-scale AI grocery assistant. We introduce a multi-faceted evaluation rubric that decomposes end-to-end shopping quality into structured dimensions and develop a calibrated LLM-as-judge pipeline aligned with human annotations. Building on this evaluation foundation, we investigate two complementary prompt-optimization strategies based on a SOTA prompt-optimizer called GEPA (Shao et al., 2025): (1) Sub-agent GEPA, which optimizes individual agent nodes against localized rubrics, and (2) MAMuT (Multi-Agent Multi-Turn) GEPA (Herrera et al., 2026), a novel system-level approach that jointly optimizes prompts across agents using multi-turn simulation and trajectory-level scoring. We release rubric templates and evaluation design guidance to support practitioners building production CSAs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
