arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2509.03312cs.CLcs.MA
收藏

AgenTracer:谁在导致LLM智能体系统发生故障?

AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?

NUS(新加坡国立大学);CUHK(香港中文大学);OPPO(OPPO公司);NTU(南洋理工大学)

Guibin Zhang, Junhao Wang, Junjie Chen, Wangchunshu Zhou, Kun Wang, Shuicheng Yan

AI总结:

针对LLM多智能体系统故障归因准确率低的问题,提出AgenTracer框架生成TracerTraj数据集,训练出轻量故障追踪器AgenTracer-8B,性能优于主流大模型,还能提升现有多智能体系统表现。

摘要:

基于大语言模型(LLM)的智能体系统通常由多个模型、复杂的工具调用和编排协议组成,其性能显著优于单体智能体。但正是这种复杂性加剧了系统的脆弱性,使其更容易发生故障。在长执行轨迹中定位导致错误的特定智能体或步骤,这一任务被定义为智能体系统故障归因。然而,当前最先进的推理LLM在应对这一挑战时仍表现极差,准确率普遍低于10%。为填补这一空白,我们提出AgenTracer,这是首个通过反事实重放和程序化故障注入对失败的多智能体轨迹进行标注的自动化框架,由此生成了经过整理的数据集TracerTraj。利用该资源,我们开发了AgenTracer-8B——这是一款经多粒度强化学习训练的轻量级故障追踪器,能够高效诊断冗长多智能体交互中的错误。在Who&When基准测试上,AgenTracer-8B的表现优于Gemini-2.5-Pro和Claude-4-Sonnet等大型专有LLM,幅度最高达18.18%,为LLM智能体故障归因树立了新标杆。更重要的是,AgenTracer-8B可为MetaGPT和MaAS等现成多智能体系统提供可执行的反馈,使其性能提升4.8%-14.2%,为智能体AI的自我修正和自我演化提供助力。

英文摘要:

Large Language Model (LLM)-based agentic systems, often comprising multiple models, complex tool invocations, and orchestration protocols, substantially outperform monolithic agents. Yet this very sophistication amplifies their fragility, making them more prone to system failure. Pinpointing the specific agent or step responsible for an error within long execution traces defines the task of agentic system failure attribution. Current state-of-the-art reasoning LLMs, however, remain strikingly inadequate for this challenge, with accuracy generally below 10%. To address this gap, we propose AgenTracer, the first automated framework for annotating failed multi-agent trajectories via counterfactual replay and programmed fault injection, producing the curated dataset TracerTraj. Leveraging this resource, we develop AgenTracer-8B, a lightweight failure tracer trained with multi-granular reinforcement learning, capable of efficiently diagnosing errors in verbose multi-agent interactions. On the Who&When benchmark, AgenTracer-8B outperforms giant proprietary LLMs like Gemini-2.5-Pro and Claude-4-Sonnet by up to 18.18%, setting a new standard in LLM agentic failure attribution. More importantly, AgenTracer-8B delivers actionable feedback to off-the-shelf multi-agent systems like MetaGPT and MaAS with 4.8-14.2% performance gains, empowering self-correcting and self-evolving agentic AI.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑