发表机构
Nanyang Technological University; National University of Singapore; CFAR, Agency for Science, Technology and Research; UNC-Chapel Hill; Carnegie Mellon University; IHPC, Agency for Science, Technology and Research（南洋理工大学; 新加坡国立大学; 新加坡科技研究局高等研究院; 北卡罗来纳大学教堂山分校; 卡内基梅隆大学; 新加坡科技研究局高性能计算研究院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究通过受控检索实验发现，工具使用主体虽能高比例识别无用结果，却很少据此停止，仅强制整合步骤能使其停止遵循证据并提升成功率。
AI 中文摘要
当主体的工具持续返回无用结果时，它应停止依赖该工具。在具有受控来源故障的检索环境中，我们分离了主体如何判断结果与它们实际采取的行动。我们比较了在主体判定为无用的结果序列较长或较短运行后，在同一步骤停止的情况；对于时钟驱动或截止日期驱动的停止，这种对比为零。在我们记录其判断的地方，我们测试的七个主体在99%至100%的情况下将故障来源的结果判定为无用，然而它们中的大多数很少基于该判断停止。提示线索改变了它们停止的时间，但未改变它们停止所依据的内容。允许凭记忆回答和推理模式可以在不考虑证据的情况下带来提前停止，明确的预算将7-8B模型的停止移至截止日期，而提示中的停止规则或调用成本最多被部分遵循。仅当执行框架强制一个整合步骤，使主体在连续五次判定为无用的结果后回答时，停止才遵循证据。该步骤提高了每个模型在故障来源上的成功率，在预算加倍时保持停止点固定，并且当主体陈述其判断时无需额外的判断调用。一项针对300个新问题的预注册复制证实了这种分离及规则的效果。
英文摘要
An agent whose tool keeps returning nothing useful should stop relying on it. In a retrieval environment with controlled source failures, we separate how agents judge results from what they do. We compare stopping at the same step after longer and shorter runs of results the agent judged useless; this contrast is zero for clock- or deadline-driven stopping. Where we record their judgments, the seven agents we test call a failing source's results useless 97-100% of the time, yet most of them rarely stop on that judgment. Prompt cues change when they stop but not what they stop on. Permission to answer from memory and a reasoning mode can bring early stops regardless of evidence, a stated budget moves the 7-8B models' stops to the deadline, and a stopping rule or call cost in the prompt is followed at most partly. Stopping follows the evidence only when the harness enforces an integration step that makes the agent answer after five consecutive results it judged useless. This step raises failing-source success for every model, keeps the stopping point fixed when the budget doubles, and needs no extra judgment call when the agent states its judgments. A pre-registered replication on 300 fresh questions confirms the dissociation and the rule's effect.
Comments37 pages, 6 figures, 28 tables. Code: https://github.com/bennidict23/judged-useless-queried-anyway
