可恢复性与验证器强度:无人值守智能体自我改进的两个可测量前提条件
Recoverability and Verifier Strength: Two Measured Preconditions for Unattended Agent Self-Improvement
AI总结:
该研究提出可测量的验证器强度与可恢复性两个前提,构建平台测试后得出仅保留旧版本且验证器强度达标时,智能体变更可无人值守运行的结论。
摘要:
智能体平台正开始改进自身的智能体:它们读取已部署智能体的执行轨迹,推导变更,由自动验证器批准,并在无人类作者的情况下发布。该循环在多大程度上可以安全地无人值守运行?我们提出了平台可测量的两个量。验证器强度σ是可用验证器(通常为LLM评判器)与可信参考验证器的一致程度;它限制了某一步骤能对智能体进行多大程度的改进。可恢复性是撤销错误变更的成本高低:撤销是否需要新的验证器批准,以及回滚能恢复多少旧行为。仅当两者都足够时,某一步骤才可无人值守运行。在我们的平台上,该规则允许提示、工具和技能编辑无人值守运行,并将权重更新保留给人类签署批准。我们的主要贡献是这两个定义及其测量方法。我们构建了一个应用该规则的平台,并通过两种方式测试:在平台自身的批准闸上,以及在τ²-bench智能体基准的受控实验中。在验证器强度方面,平台的LLM评判器有时无论展示什么都给出相同分数(37个设置中的9个)且未报告错误,因此验证器可能看似正常却批准任何内容;必须测量其强度,且两个简单测试可发现该故障。在可恢复性方面,恢复保存的副本使智能体回到原始行为,但重新生成的撤销则未做到,因此仅保留旧版本的变更可在无人类参与的情况下撤销;且在变更部署期间,它改变了17%任务的结果,超出了正常的运行间变异,而任何撤销都无法修复。这些共同支持该规则:仅当变更的旧版本被保留且其验证器的强度已被测量时,变更才可无人值守运行。
英文摘要:
Agent platforms are starting to improve their own agents: they read a deployed agent's execution traces, derive a change, have an automated verifier approve it, and release it with no human author. How much of this loop can safely run without a human? We propose two quantities a platform can measure. Verifier strength $σ$ is how well the available verifier -- often an LLM judge -- agrees with a trusted reference verifier; it limits how much a step can improve the agent. Recoverability is how cheaply a bad change can be undone: whether undoing it needs a new verifier approval, and how much of the old behavior a rollback restores. A step should run unattended only if both are adequate. On our platform this rule allows prompt, tool and skill edits to run unattended and keeps weight updates under human sign-off. Our main contribution is these two definitions and how to measure them. We built a platform that applies the rule and tested it in two ways: on the platform's own approval gate, and in controlled experiments on the $τ^2$-bench agent benchmark. On verifier strength, the platform's LLM judge sometimes gave the same score no matter what it was shown (9 of 37 settings) and reported no error, so a verifier can look healthy while approving anything; its strength must be measured, and two simple tests catch the failure. On recoverability, restoring a saved copy brought the agent back to its original behavior, but a regenerated undo did not, so only changes whose old version is kept can be reversed without a human; and while a change was deployed, it altered the outcome of 17 percent of tasks beyond normal run-to-run variation, which no undo repairs. Together these support the rule: a change can run unattended only when its old version is kept and its verifier's strength has been measured.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。