从特征到行动：传统与智能体AI系统中的可解释性
From Features to Actions: Explainability in Traditional and Agentic AI Systems
AI总结：
本文比较了基于归因的解释与基于轨迹的诊断在静态和智能体设置中的效果，发现归因方法无法可靠诊断智能体轨迹中的执行级故障，而轨迹级可解释性更能定位行为故障。
摘要：
在过去十年中，可解释AI主要关注解释单个模型预测，在固定决策结构下生成将输入与输出关联的事后解释。大型语言模型的最新进展使得智能体AI系统能够在多步轨迹中展开行为。在这些设置中，成功与失败由决策序列而非单个输出决定。目前尚不清楚为静态预测设计的解释方法如何应用于行为随时间涌现的智能体设置。在这项工作中，我们通过比较两种设置中基于归因的解释与基于轨迹的诊断来弥合这一差距。我们的结果表明，虽然归因方法在静态设置中实现了稳定的特征排名（Spearman ρ = 0.86），但它们无法可靠地诊断智能体轨迹中的执行级故障。相比之下，针对智能体设置的轨迹接地评分标准能够一致地定位行为故障，并揭示状态跟踪不一致在失败运行中的普遍性高出2.7倍，并将成功概率降低49%。这些发现促使我们转向轨迹级可解释性，以评估和诊断智能体系统中自主AI行为。代码：https://github.com/VectorInstitute/unified-xai-evaluation-framework 项目页面：https://vectorinstitute.github.io/unified-xai-evaluation-framework
英文摘要：
Over the last decade, Explainable AI has primarily focused on interpreting individual model predictions, producing post-hoc explanations that relate inputs to outputs under a fixed decision structure. Recent advances in large language models (LLMs) have enabled agentic AI systems whose behaviour unfolds over multi-step trajectories. In these settings, success and failure are determined by sequences of decisions rather than a single output. It remains unclear how explanation approaches designed for static predictions translate to agentic settings where behaviour emerges over time. In this work, we bridge this gap by comparing attribution-based explanations with trace-based diagnostics across both settings. Our results show that while attribution methods achieve stable feature rankings in static settings (Spearman \r{ho} = 0.86), they cannot be applied reliably to diagnose execution-level failures in agentic trajectories. In contrast, trace-grounded rubric evaluation for agentic settings consistently localizes behaviour breakdowns and reveals that state tracking inconsistency is 2.7x more prevalent in failed runs and reduces success probability by 49%. These findings motivate a shift towards trajectory-level explainability for evaluating and diagnosing autonomous AI behaviour in agentic systems. Code: https://github.com/VectorInstitute/unified-xai-evaluation-framework Project page: https://vectorinstitute.github.io/unified-xai-evaluation-framework
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
