arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.04306cs.CL
收藏

ScoreFlow:通过基于分数的偏好优化掌握LLM智能体工作流

ScoreFlow: Mastering LLM Agent Workflows via Score-based Preference Optimization

University of Chicago(芝加哥大学);Princeton University(普林斯顿大学);University of Oxford(牛津大学)

Yinjie Wang, Ling Yang, Guohao Li, Mengdi Wang, Bryon Aragam

AI总结:

ScoreFlow提出基于分数的直接偏好优化变体Score-DPO,在连续空间中进行梯度优化以自动构建LLM多智能体工作流,在六个基准上提升8.2%,并让小模型以更低推理成本超越大模型。

摘要:

近期研究利用大语言模型多智能体系统解决复杂问题,同时试图减少构建这些系统所需的人工工作,推动了自动化智能体工作流优化方法的发展。然而,现有方法在依赖离散优化技术时,由于表示能力受限、缺乏适应性以及可扩展性差,仍然缺乏灵活性。我们通过ScoreFlow应对这些挑战,这是一个简单而高性能的框架,利用连续空间中高效的基于梯度的优化。ScoreFlow融合了Score-DPO,这是直接偏好优化方法的一种新颖变体,能够考虑定量反馈。在涵盖问答、编码和数学推理的六个基准测试中,ScoreFlow相比现有基线实现了8.2%的提升。此外,它使较小的模型能够以更低的推理成本超越较大的模型。项目地址:https://github.com/Gen-Verse/ScoreFlow

英文摘要:

Recent research has leveraged large language model multi-agent systems for complex problem-solving while trying to reduce the manual effort required to build them, driving the development of automated agent workflow optimization methods. However, existing methods remain inflexible due to representational limitations, a lack of adaptability, and poor scalability when relying on discrete optimization techniques. We address these challenges with ScoreFlow, a simple yet high-performance framework that leverages efficient gradient-based optimization in a continuous space. ScoreFlow incorporates Score-DPO, a novel variant of the direct preference optimization method that accounts for quantitative feedback. Across six benchmarks spanning question answering, coding, and mathematical reasoning, ScoreFlow achieves an 8.2% improvement over existing baselines. Moreover, it empowers smaller models to outperform larger ones with lower inference costs. Project: https://github.com/Gen-Verse/ScoreFlow

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑