arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.00623cs.AIcs.CL
收藏

TraceSIR: 一个用于结构化分析和报告代理执行轨迹的多代理框架

TraceSIR: A Multi-Agent Framework for Structured Analysis and Reporting of Agentic Execution Traces

Beijing Institute of Technology(北京理工大学);Zhipu AI(智谱AI);Tsinghua University(清华大学);Shanghai Jiao Tong University(上海交通大学)
Shu-Xun Yang, Cunxiang Wang, Haoke Zhang, Wenbo Yu, Lindong Wu, Jiayi Gui, Dayong Yang, Yukuo Cen, Zhuoer Feng, Bosi Wen, Yidong Wang, Lucen Zhong, Jiamin Ren, … 展开作者

Shu-Xun Yang, Cunxiang Wang, Haoke Zhang, Wenbo Yu, Lindong Wu, Jiayi Gui, Dayong Yang, Yukuo Cen, Zhuoer Feng, Bosi Wen, Yidong Wang, Lucen Zhong, Jiamin Ren, Linfeng Zhang, Jie Tang

AI总结:

TraceSIR通过多代理框架结构化分析和报告代理执行轨迹,提升故障诊断和根本原因分析效率。

摘要:

代理系统通过将大型语言模型与外部工具和迭代决策相结合,能够执行复杂任务,如深度研究、函数调用和编码。然而,其长而复杂的执行轨迹使得故障诊断和根本原因分析变得极其具有挑战性。手动检查无法扩展,而直接对原始轨迹应用LLM则受到输入长度限制和不可靠推理的阻碍。仅关注最终任务结果会丢弃用于准确问题定位所需的关键行为信息。为了解决这些问题,我们提出了TraceSIR,一个用于结构化分析和报告代理执行轨迹的多代理框架。TraceSIR协调三个专门的代理:(1)StructureAgent,它引入了一种新的抽象格式,TraceFormat,以压缩执行轨迹同时保留关键行为信息;(2)InsightAgent,它执行细粒度诊断,包括问题定位、根本原因分析和优化建议;(3)ReportAgent,它汇总跨任务实例的洞察并生成综合分析报告。为了评估TraceSIR,我们构建了TraceBench,涵盖三个现实世界的代理场景,并引入ReportEval,一种评估分析报告质量和实用性的评估协议,以符合行业需求。实验表明,TraceSIR始终产生连贯、信息丰富且可操作的报告,在所有评估维度上均显著优于现有方法。我们的项目和视频可在https://github.com/SHU-XUN/TraceSIR上公开获取。

英文摘要:

Agentic systems augment large language models with external tools and iterative decision making, enabling complex tasks such as deep research, function calling, and coding. However, their long and intricate execution traces make failure diagnosis and root cause analysis extremely challenging. Manual inspection does not scale, while directly applying LLMs to raw traces is hindered by input length limits and unreliable reasoning. Focusing solely on final task outcomes further discards critical behavioral information required for accurate issue localization. To address these issues, we propose TraceSIR, a multi-agent framework for structured analysis and reporting of agentic execution traces. TraceSIR coordinates three specialized agents: (1) StructureAgent, which introduces a novel abstraction format, TraceFormat, to compress execution traces while preserving essential behavioral information; (2) InsightAgent, which performs fine-grained diagnosis including issue localization, root cause analysis, and optimization suggestions; (3) ReportAgent, which aggregates insights across task instances and generates comprehensive analysis reports. To evaluate TraceSIR, we construct TraceBench, covering three real-world agentic scenarios, and introduce ReportEval, an evaluation protocol for assessing the quality and usability of analysis reports aligned with industry needs. Experiments show that TraceSIR consistently produces coherent, informative, and actionable reports, significantly outperforming existing approaches across all evaluation dimensions. Our project and video are publicly available at https://github.com/SHU-XUN/TraceSIR.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑