CausalFlow: LLM Agent 失败的因果归因与反事实修复
CausalFlow: Causal Attribution and Counterfactual Repair for LLM Agent Failures
AI总结：
提出CausalFlow框架，通过反事实干预计算步骤级因果责任分数，识别失败步骤并生成最小编辑修复，用于测试时修复和训练时监督，在多个基准上优于启发式方法。
摘要：
大型语言模型（LLM）代理在涉及推理、工具使用和环境交互的多步任务中经常失败。虽然此类失败通常被记录或通过启发式重试处理，但它们包含了关于执行中断位置的结构化信号。我们提出了CausalFlow，一个干预框架，将失败的代理轨迹转换为最小的反事实修复和可重用的监督。CausalFlow将执行轨迹建模为依赖步骤的顺序链，并通过步骤级反事实干预计算因果责任分数（CRS）来识别导致失败的步骤。对于这些步骤，我们生成最小编辑修复，将最终结果翻转为成功，产生形式为（错误步骤，修正步骤）的验证对比对。CausalFlow支持两种互补用途：具有最小行为漂移的针对性测试时修复，以及适用于离线偏好优化或奖励建模的训练时监督。在涵盖数学推理、代码生成、问答和医学浏览的四个基准测试中，CausalFlow将失败执行转换为具有高最小性和因果一致性分数的验证最小修复，并证明因果归因对于跨不同代理任务的可靠改进是必要的，在复杂检索设置中优于启发式细化，同时产生更局部的修复。这些结果表明，对结构化执行轨迹的干预分析提供了一种原则性和可扩展的机制，将代理失败转化为可靠性提升和可学习的监督。
英文摘要：
Large language model (LLM) agents frequently fail on multi-step tasks involving reasoning, tool use, and environment interaction. While such failures are typically logged or retried heuristically, they contain structured signals about where execution broke down. We introduce CausalFlow, an interventional framework that converts failed agent traces into minimal counterfactual repairs and reusable supervision. CausalFlow models execution traces as sequential chains of dependent steps and computes Causal Responsibility Scores(CRS) via step-level counterfactual intervention to identify failure-inducing steps. For these steps, we generate minimally edited repairs that flip the final outcome to success, producing validated contrastive pairs of the form (wrong step, corrected step). CausalFlow supports two complementary uses: targeted test-time repair that recovers from failures with minimal behavioral drift, and training-time supervision suitable for offline preference optimization or reward modeling. Across four benchmarks spanning mathematical reasoning, code generation, question answering, and medical browsing, CausalFlow converts failed executions into validated minimal repairs with high minimality and causal-consensus scores, and demonstrates that causal attribution is necessary for reliable improvement across diverse agent tasks, outperforming heuristic refinement in complex retrieval settings while producing more localized repairs throughout. These results demonstrate that interventional analysis over structured execution traces provides a principled and scalable mechanism for transforming agent failures into reliability gains and learning-ready supervision.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
