arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.22480cs.AIcs.CLcs.LG
收藏

VeRO: 用于优化智能体的智能体框架

VeRO: A Harness for Agents to Optimize Agents

Varun Ursekar, Apaar Shanker, Veronica Chatrath, Yuan Xue, Samuel Marc Denton

AI总结:

提出 VeRO 框架和 VeRO-Bench 基准,通过版本化快照、预算控制评估和结构化执行轨迹来优化智能体代码,并实验比较不同优化器对目标智能体的改进效果。

摘要:

编码智能体的一个重要新兴应用是智能体框架优化:通过编辑和评估目标智能体的代码来迭代改进它。尽管具有相关性,但社区对编码智能体在此任务上的表现缺乏系统理解。框架优化与传统软件工程不同:智能体框架将确定性代码与随机 LLM 完成交错,需要结构化捕获中间执行轨迹和下游结果。为了解决这些挑战,我们引入了 (1) VeRO(版本化、奖励和观察),一个外部框架,提供目标框架的版本化快照、预算控制评估和结构化执行轨迹,以及 (2) VeRO-Bench,一个包含参考评估程序的目标智能体和任务的基准套件。使用 VeRO,我们进行了一项实证研究,比较了不同任务上的优化器,并分析了哪些修改能可靠地改进目标智能体框架。我们发布 VeRO 以支持作为编码智能体核心能力的智能体优化研究。代码可在 https://github.com/scaleapi/vero 获取。

英文摘要:

An important emerging application of coding agents is agent harness optimization: the iterative improvement of a target agent by editing and evaluating its code. Despite its relevance, the community lacks a systematic understanding of coding agent performance on this task. Harness optimization differs from conventional software engineering: agent harnesses interleave deterministic code with stochastic LLM completions, requiring structured capture of both intermediate execution traces and downstream outcomes. To address these challenges, we introduce (1) VeRO (Versioning, Rewards, and Observations), an outer harness that provides versioned snapshots, budget-controlled evaluation, and structured execution traces of target harnesses, and (2) VeRO-Bench, a benchmark suite of target agents and tasks with reference evaluation procedures. Using VeRO, we conduct an empirical study comparing optimizers across tasks and analyzing which modifications reliably improve target agent harnesses. We release VeRO to support research on agent optimization as a core capability for coding agents. Code is available at https://github.com/scaleapi/vero.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑