arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.22779cs.AI
收藏

你何时能信任智能体自身的验证器?在搜索压力与神谕访问下对反事实理由验证的压力测试

When Can You Trust an Agent's Own Verifier? Stress-Testing Counterfactual Rationale Verification Under Search Pressure and Oracle Access

Yonsei University(延世大学)

Jonghyun Lee, Moonsu Kim, Sunghwan Park

AI总结:

该研究提出一种压力测试方法,通过压力比率ρ和四个探测项测试反事实理由验证器的可靠性,将其应用于RCS验证器,发现其存在多种失效模式,建议审计验证器后再使用。

摘要:

自动化智能体设计越来越依赖单一分数来判断一项改动是否为改进,但经过足够激进优化的分数会不再衡量其原本要衡量的内容。当智能体同时解释其设计时,该解释是我们无需人工标注即可检查的第二个信号:否定每个陈述的主张、重新运行并观察性能是否下降。我们探究当智能体针对该验证器进行优化后,此类验证器是否仍保持可靠。我们的贡献并非提出新的验证器,而是一种对其进行压力测试的方法:一个衡量验证器引导智能体搜索程度的压力比率ρ,以及四个可证伪的探测项,涵盖四种失效模式——无信号、强度过弱无影响、通过陈述更少主张被操控、在智能体做出承诺前被查询。我们将该测试应用于我们自身的验证器——理由一致性分数(RCS),该验证器内嵌于一个设计机器学习流水线的大语言模型(LLM)智能体中,使用1个生成模型、6个任务和3个随机种子。首先,在我们先前的配置中,该验证器几乎不影响搜索目标(ρ远小于1),部分原因在于存在一个准确率门限,因此其鲁棒性未经过测试;它仍通过最终选择的平局决胜来塑造所报告的产物,而这一点未被ρ记录。其次,一项能力控制实验显示,当指示智能体缩短其主张列表时,搜索过程并未改变,即便ρ超过先前水平的100倍;由于反馈省略了主张数量且搜索范围较短,该零结果在很大程度上是结构性的,而非鲁棒性的证据。第三,当智能体能在做出承诺前探测验证器时,它会保留通过验证的主张,而保留的忠实度却下降——这种下降在循环内指标中不可见,尽管部分与RCS对小型流水线的评分方式存在混淆。我们将这些作为初步证据,并提出一项报告标准:在信任验证器前,需用ρ和这些探测项对其进行审计。

英文摘要:

Automated agent design increasingly trusts a single score to decide whether a change is an improvement -- but a score optimized aggressively enough ceases to measure what it was meant to. When an agent also explains its design, that explanation is a second signal we can check without human labels: negate each stated claim, re-run, and observe whether performance drops. We ask whether such a verifier stays reliable once an agent optimizes against it. Our contribution is not a new verifier but a way to stress-test one: a pressure ratio $ρ$ measuring how much the verifier steers the agent's search, and four falsifiable probes covering four failure modes -- giving no signal, being too weak to matter, being gamed by stating fewer claims, and being queried before the agent commits. We apply the test to our own verifier, the Rationale Consistency Score (RCS), inside an LLM agent that designs machine-learning pipelines, with one generator model, six tasks, and three seeds. First, in our prior configuration the verifier barely affects the search objective ($ρ\ll 1$), partly because of an accuracy gate, so its robustness had not been tested; it still shapes the reported artifact through a final-selection tie-break that $ρ$ does not register. Second, a capability control shows the agent shortens its claim lists when instructed, yet search does not, even at $ρ$ over 100 times the prior level; since feedback omits claim volume and horizons are short, this null result is largely structural rather than evidence of robustness. Third, when the agent can probe the verifier before committing, it keeps claims that pass while held-out faithfulness falls -- a decline invisible to in-loop metrics, though partly confounded by how RCS scores small pipelines. We offer these as preliminary evidence and propose a reporting standard: audit a verifier with $ρ$ and these probes before trusting it.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑