发表机构
Peking University; Joy Future Academy; Tsinghua University（北京大学; 京东探索研究院; 清华大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对基于LLM的智能体系统故障难以追溯至早期根源步骤的问题，提出ReCast方法，通过选择相关层、构建互补特征及训练编码器学习归因导向步骤表示，在四个基准测试中取得最优Hit@1指标。
AI 中文摘要
在基于大语言模型（LLM）的智能体系统中，故障可能源于早期步骤，其影响会在后续交互中传播，导致故障起源难以识别。为将此类故障追溯至其起源，故障归因被定义为识别导致故障的最早步骤的任务。近期方法利用LLM内部信号进行故障归因，通常将隐藏状态作为步骤表示。因此，我们开展实证研究以评估这些表示区分根本原因步骤与其他步骤的有效性，发现其区分度有限。受此观察启发，我们提出ReCast，一种步骤表示学习方法，将冻结LLM的隐藏状态转换为面向归因的步骤表示。ReCast首先选择与归因相关的层，接着构建互补的模式特征与偏差特征，最后通过采用对比损失和排序损失训练的编码器学习上下文感知的步骤表示。我们还引入了用于故障归因的训练数据集ReCast-2K。ReCast在四个基准测试中取得了最佳的Hit@1指标，在Who&When Algorithm基准上超越最强基线5.65个百分点，在Handcrafted基准上超越最强基线9.19个百分点。代码可在指定URL获取。
英文摘要
In LLM-based agent systems, failures can originate from early steps whose effects propagate through subsequent interactions, making their origins difficult to identify. To trace such failures back to their origin, failure attribution has been formulated as the task of identifying the earliest step responsible for the failure. Recent methods leverage LLM internal signals for failure attribution, typically using hidden states as step representations. We therefore conduct an empirical study to evaluate how effectively these representations distinguish root-cause steps from other steps and find limited separation. Motivated by this observation, we propose ReCast, a step representation learning method that transforms hidden states from a frozen LLM into attribution-oriented step representations. ReCast first selects attribution-relevant layers, then constructs complementary pattern and deviation features, and finally learns contextualized step representations through an encoder trained with contrastive and ranking objectives. We also introduce ReCast-2K, a training dataset for failure attribution. ReCast achieves the best Hit@1 across four benchmarks, surpassing the strongest baseline by 5.65 and 9.19 pp on Who&When Algorithm and Handcrafted, respectively. Code is available at https://anonymous.4open.science/r/ReCast-5FB6 .
