可恢复性有定律:工具增强智能体的ERR度量
Recoverability Has a Law: The ERR Measure for Tool-Augmented Agents
AI总结:
本文提出了一种预测理论,通过预期恢复遗憾(ERR)与效率分数(ES)的关系,揭示了语言模型智能体在工具使用中的可恢复性遵循可测量的定律。
摘要:
语言模型智能体在工具调用执行失败后常常表现出自我恢复的能力,但这种行为缺乏正式的解释。我们提出了一种预测理论,通过展示可恢复性遵循可测量的定律来填补这一空白。具体而言,我们通过预期恢复遗憾(ERR)来形式化可恢复性,该指标量化了在随机执行噪声下恢复策略与最优策略之间的偏差,并推导出ERR与一个经验可观测量,即效率分数(ES)之间的第一阶关系。这产生了一个可检验的第一阶恢复动力学定量定律。我们在五个工具使用基准上实证验证了该定律,涵盖受控扰动、诊断推理和现实世界API。在模型规模、扰动范围和恢复时间范围内,根据ERR-ES定律预测的遗憾与从蒙特卡洛回滚中测量的失败后遗憾的差异小于等于0.05。我们的结果表明,可恢复性不是模型规模或架构的产物,而是在交互动力学中的受控属性,为语言智能体在执行层面的鲁棒性提供了理论基础。
英文摘要:
Language model agents often appear capable of self-recovery after failing tool call executions, yet this behavior lacks a formal explanation. We present a predictive theory that resolves this gap by showing that recoverability follows a measurable law. To elaborate, we formalize recoverability through Expected Recovery Regret (ERR), which quantifies the deviation of a recovery policy from the optimal one under stochastic execution noise, and derive a first-order relationship between ERR and an empirical observable quantity, the Efficiency Score (ES). This yields a falsifiable first-order quantitative law of recovery dynamics in tool-using agents. We empirically validate the law across five tool-use benchmarks spanning controlled perturbations, diagnostic reasoning, and real-world APIs. Across model scales, perturbation regimes, and recovery horizons, predicted regret under the ERR-ES law closely matched observed post-failure regret measured from Monte Carlo rollouts, within delta less than or equal to 0.05. Our results reveal that recoverability is not an artifact of model scale or architecture, but a governed property of interaction dynamics, providing a theoretical foundation for execution-level robustness in language agents.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。