JudgeFlow: 通过块判断实现代理工作流优化
JudgeFlow: Agentic Workflow Optimization via Block Judge
AI总结：
JudgeFlow通过引入可配置的逻辑块和专用判断模块，实现代理工作流的高效优化，提升样本效率和可解释性，并在数学推理和代码生成任务中表现更优。
摘要：
优化基于大语言模型的代理工作流在扩展AI能力时具有挑战性。当前方法依赖于粗粒度的端到端评估信号，缺乏对在哪里进行细化的细粒度信号，往往导致低效或低影响的修改。为了解决这些限制，我们提出了JudgeFlow，一个评估-判断-优化-更新的流水线。我们将在代理工作流中加入可重用、可配置的逻辑块，以捕捉基本的逻辑形式。在此抽象之上，我们设计了一个专用的判断模块，该模块特别检查执行轨迹，尤其是失败的运行，并为有问题的块分配基于排名的责任分数。这些细粒度的诊断信号随后由基于LLM的优化器利用，该优化器将修改集中于工作流中最有问题的块。我们的方法提高了样本效率，通过块级诊断增强了可解释性，并为自动化日益复杂代理工作流提供了可扩展的基础。我们在数学推理和代码生成基准上评估了JudgeFlow，其中JudgeFlow在性能和效率上均优于现有方法。
英文摘要：
Optimizing LLM-based agentic workflows is challenging for scaling AI capabilities. Current methods rely on coarse, end-to-end evaluation signals and lack fine-grained signals on where to refine, often resulting in inefficient or low-impact modifications. To address these limitations, we propose JudgeFlow, an Evaluation-Judge-Optimization-Update pipeline. We incorporate reusable, configurable logic blocks into agentic workflows to capture fundamental forms of logic. On top of this abstraction, we design a dedicated Judge module that inspects execution traces particularly failed runs and assigns rank-based responsibility scores to problematic blocks. These fine-grained diagnostic signals are then leveraged by an LLM-based optimizer, which focuses modifications on the most problematic block in the workflow. Our approach improves sample efficiency, enhances interpretability through block-level diagnostics, and provides a scalable foundation for automating increasingly complex agentic workflows. We evaluate JudgeFlow on mathematical reasoning and code generation benchmarks, where JudgeFlow achieves superior performance and efficiency compared to existing methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
