REFLECT: 针对LLM智能体轨迹中静默失败的干预支持错误归因
REFLECT: Intervention-Supported Error Attribution for Silent Failures in LLM Agent Traces
AI总结：
提出REFLECT方法，通过诊断候选错误步骤、使用诊断特定补丁进行受控重放测试，并利用验证结果作为对比证据来细化归因，在四个基准上取得最高定位准确率。
摘要：
大型语言模型（LLM）智能体现在通过长时间的计划与执行轨迹来解决复杂任务，但在已完成轨迹中定位错误的能力仍然远远落后，尤其是在静默失败情况下。现有方法通过分类器或LLM法官预测可疑步骤，或通过重试恢复正确答案，但都没有将干预结果反馈回来以细化归因本身。我们提出REFLECT方法，通过诊断候选错误步骤，使用诊断特定补丁进行受控重放测试，并利用验证的结果翻转作为对比证据来细化最终归因，从而弥合这一差距。在跨越领域多跳推理的四个定位基准上，REFLECT在所有四个基准中均实现了同审计方法中最高的定位准确率，在结构化工具使用轨迹上取得了最大增益，并且在无法获得真实答案时也能提供可操作的定位。
英文摘要：
Large language model (LLM) agents now solve complex tasks through long plan-and-execution traces, yet the ability to locate errors in a completed traces still lags far behind, especially in the \emph{silent failure} regime. Existing approaches predict suspect steps via classifiers or LLM judges, or recover correct answers via retry, but none feed the intervention outcome back to \emph{refine the attribution itself}. We propose \methodname, a method that closes this gap by diagnosing a candidate error step, testing it through controlled replay with a diagnosis-specific patch, and using the verified outcome flip as contrastive evidence to refine the final attribution. Across four localization benchmarks spanning multi-hop reasoning across domains, \methodname achieves the highest localization accuracy among same-auditor methods across all four benchmarks, with the largest gains on structured tool-use traces, while providing actionable localization even when ground-truth answers are unavailable.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
