因果智能体回放：LLM智能体故障的反事实归因
Causal Agent Replay: Counterfactual Attribution for LLM-Agent Failures
AI总结：
提出Causal Agent Replay (CAR)方法，通过结构因果模型和干预操作，对LLM智能体失败步骤进行反事实归因，解决现有方法无法定位决策步骤的问题。
摘要：
当LLM智能体失败时——例如发放了不应发放的退款、调用了错误的工具、泄露了数据——现有工具只能回答发生了什么（可观测性）或是否通过（评估），但无法回答哪个步骤导致了失败。直观的启发式方法是错误的：执行有害动作的步骤通常不是决定该动作的步骤，而LLM判断的归因是相关性的且不可靠（在Who&When基准上，最先进的步骤级准确率约为14%）。我们提出Causal Agent Replay (CAR)，通过干预来回答这个问题：它将智能体运行建模为结构因果模型，对某个步骤应用do操作，并在相同随机策略下重新执行轨迹，测量结果分布的变化。我们定义了智能体步骤上的干预代数、一个单步对比估计器（其承诺点规则解决了特定于随机向前运行的混杂因素），以及一个预算有界的蒙特卡洛Shapley估计器（用于在交互步骤间分配信用）。每个效应都附有置信区间。我们在具有植入真实标签的合成结构因果模型上进行验证：对比估计器恢复了关键步骤，Shapley恢复了两步交互（0.44, 0.45, ~0；效率总和0.909对比解析值0.91）。CAR是开源的，可在托管或免费的本地模型上运行。
英文摘要：
When an LLM agent fails -- issues a refund it should not have, calls the wrong tool, leaks data -- existing tooling answers what happened (observability) or whether it passed (evaluation), but not which step caused the failure. The obvious heuristics are wrong: the step that executes the harmful action is usually not the step that decided on it, and LLM-judge attribution is correlational and unreliable (state-of-the-art step-level accuracy on the Who&When benchmark is about 14%). We present Causal Agent Replay (CAR), which answers the question by intervention: it models an agent run as a structural causal model, applies a do-operation to a step, and re-executes the trajectory forward under the same stochastic policy, measuring the shift in the outcome distribution. We define an intervention algebra over agent steps, a single-step contrastive estimator whose point-of-commitment rule resolves a confound specific to stochastic run-forward, and a budget-bounded Monte-Carlo Shapley estimator that splits credit across interacting steps. Every effect is reported with confidence intervals. We validate against synthetic structural causal models with planted ground truth: the contrastive estimator recovers the pivotal step, and Shapley recovers a two-step interaction (0.44, 0.45, ~0; efficiency sum 0.909 versus the analytic 0.91). CAR is open source and runs on hosted or free local models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
