代理中准确的失败预测并不意味着有效的失败预防
Accurate Failure Prediction in Agents Does Not Imply Effective Failure Prevention
AI总结：
研究发现LLM批评模型的高准确度并不保证安全干预，提出预部署测试以判断干预是否有益，避免部署前的严重性能下降。
摘要：
LLM批评模型的主动干预通常被认为能提高可靠性，但其在部署时的效果却知之甚少。我们展示了一个二元LLM批评模型，其离线准确度（AUROC 0.94）很高，但仍然会导致严重性能下降，在一个模型上导致26个百分点的崩溃，而在另一个模型上影响接近零个百分点。这种变化表明，仅凭LLM批评模型的准确度不足以确定干预是否安全。我们识别出了一种中断-恢复权衡：干预可能恢复失败轨迹，但也可能破坏本应成功的轨迹。基于这一见解，我们提出了一种预部署测试，使用50个任务的小型试点来估计干预是否可能帮助或损害，而无需完全部署。在多个基准测试中，该测试正确预测了结果：干预在高成功率任务上降低性能（0至-26个百分点），而在高失败ALFWorld基准上产生小幅提升（+2.8个百分点，p=0.014）。因此，我们框架的主要价值在于确定何时不进行干预，从而在部署前防止严重的退化。
英文摘要：
Proactive interventions by LLM critic models are often assumed to improve reliability, yet their effects at deployment time are poorly understood. We show that a binary LLM critic with strong offline accuracy (AUROC 0.94) can nevertheless cause severe performance degradation, inducing a 26 percentage point (pp) collapse on one model while affecting another by near zero pp. This variability demonstrates that LLM critic accuracy alone is insufficient to determine whether intervention is safe. We identify a disruption-recovery tradeoff: interventions may recover failing trajectories but also disrupt trajectories that would have succeeded. Based on this insight, we propose a pre-deployment test that uses a small pilot of 50 tasks to estimate whether intervention is likely to help or harm, without requiring full deployment. Across benchmarks, the test correctly anticipates outcomes: intervention degrades performance on high-success tasks (0 to -26 pp), while yielding a modest improvement on the high-failure ALFWorld benchmark (+2.8 pp, p=0.014). The primary value of our framework is therefore identifying when not to intervene, preventing severe regressions before deployment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
