溯因、行动、预测:为多智能体系统中的自动化失败归因搭建因果推理脚手架
Abduct, Act, Predict: Scaffolding Causal Inference for Automated Failure Attribution in Multi-Agent Systems
AI总结:
本文提出A2P Scaffolding框架,通过溯因、行动、预测三步因果推理替代传统模式识别,将多智能体失败归因的步骤级准确率提升约2.4至2.9倍。
摘要:
多智能体系统中的失败归因——精确定位决定性错误发生的具体步骤——是一个关键但尚未解决的挑战。当前方法将其视为对长对话日志的模式识别任务,导致步骤级准确率极低(低于17%),使其在调试复杂系统时不具备实用性。这些方法的核心弱点在于从根本上缺乏稳健的反事实推理能力:即无法判断纠正单个动作是否真的能够避免任务失败。为弥合这一反事实推理鸿沟,我们提出了Abduct-Act-Predict(A2P)Scaffolding,这是一种新颖的智能体框架,将失败归因从模式识别转变为结构化的因果推理任务。A2P在单次推理过程中明确引导大语言模型完成一个形式化的三步推理流程:(1)溯因(Abduction),推断智能体行为背后的隐藏根本原因;(2)行动(Action),定义一个最小化的纠正干预;(3)预测(Prediction),模拟后续轨迹并验证该干预是否解决了失败。这种结构化方法利用了整个对话的整体上下文,同时对模型的分析施加了严格的因果逻辑。我们在Who&When基准上的大量实验证明了其有效性。在Algorithm-Generated数据集上,A2P实现了47.46%的步骤级准确率,比基线的16.67%提高了2.85倍。在更复杂的Hand-Crafted数据集上,它实现了29.31%的步骤准确率,比基线的12.07%提高了2.43倍。通过从因果视角重新定义问题,A2P Scaffolding为自动化失败归因提供了一种稳健、可验证且准确率显著更高的解决方案。我们的代码已发布在https://github.com/ResearAI/A2P。
英文摘要:
Failure attribution in multi-agent systems -- pinpointing the exact step where a decisive error occurs -- is a critical yet unsolved challenge. Current methods treat this as a pattern recognition task over long conversation logs, leading to critically low step-level accuracy (below 17\%), which renders them impractical for debugging complex systems. Their core weakness is a fundamental inability to perform robust counterfactual reasoning: to determine if correcting a single action would have actually averted the task failure. To bridge this \emph{counterfactual inference gap}, we introduce Abduct-Act-Predict (A2P) Scaffolding, a novel agent framework that transforms failure attribution from pattern recognition into a structured causal inference task. A2P explicitly guides a large language model through a formal three-step reasoning process within a single inference pass: (1) Abduction, to infer the hidden root causes behind an agent's actions; (2) Action, to define a minimal corrective intervention; and (3) Prediction, to simulate the subsequent trajectory and verify if the intervention resolves the failure. This structured approach leverages the holistic context of the entire conversation while imposing a rigorous causal logic on the model's analysis. Our extensive experiments on the Who\&When benchmark demonstrate its efficacy. On the Algorithm-Generated dataset, A2P achieves 47.46\% step-level accuracy, a 2.85$\times$ improvement over the 16.67\% of the baseline. On the more complex Hand-Crafted dataset, it achieves 29.31\% step accuracy, a 2.43$\times$ improvement over the baseline's 12.07\%. By reframing the problem through a causal lens, A2P Scaffolding provides a robust, verifiable, and significantly more accurate solution for automated failure attribution. Ours code are released at https://github.com/ResearAI/A2P.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。