ScoreFlow:通过基于分数的偏好优化掌握LLM智能体工作流
ScoreFlow: Mastering LLM Agent Workflows via Score-based Preference Optimization
AI总结:
ScoreFlow提出基于分数的直接偏好优化变体Score-DPO,在连续空间中进行梯度优化以自动构建LLM多智能体工作流,在六个基准上提升8.2%,并让小模型以更低推理成本超越大模型。
摘要:
近期研究利用大语言模型多智能体系统解决复杂问题,同时试图减少构建这些系统所需的人工工作,推动了自动化智能体工作流优化方法的发展。然而,现有方法在依赖离散优化技术时,由于表示能力受限、缺乏适应性以及可扩展性差,仍然缺乏灵活性。我们通过ScoreFlow应对这些挑战,这是一个简单而高性能的框架,利用连续空间中高效的基于梯度的优化。ScoreFlow融合了Score-DPO,这是直接偏好优化方法的一种新颖变体,能够考虑定量反馈。在涵盖问答、编码和数学推理的六个基准测试中,ScoreFlow相比现有基线实现了8.2%的提升。此外,它使较小的模型能够以更低的推理成本超越较大的模型。项目地址:https://github.com/Gen-Verse/ScoreFlow
英文摘要:
Recent research has leveraged large language model multi-agent systems for complex problem-solving while trying to reduce the manual effort required to build them, driving the development of automated agent workflow optimization methods. However, existing methods remain inflexible due to representational limitations, a lack of adaptability, and poor scalability when relying on discrete optimization techniques. We address these challenges with ScoreFlow, a simple yet high-performance framework that leverages efficient gradient-based optimization in a continuous space. ScoreFlow incorporates Score-DPO, a novel variant of the direct preference optimization method that accounts for quantitative feedback. Across six benchmarks spanning question answering, coding, and mathematical reasoning, ScoreFlow achieves an 8.2% improvement over existing baselines. Moreover, it empowers smaller models to outperform larger ones with lower inference costs. Project: https://github.com/Gen-Verse/ScoreFlow
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。