发表机构
Mälardalen University; Westermo Network Technologies AB（马勒达尔大学; 韦斯特莫网络技术有限公司）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究考察工具增强的大语言模型系统在工业夜间测试失败根因分析中的应用，实现单智能体与多智能体两种配置，通过真实失败场景和从业者评估发现单智能体在成本与速度上更优，多智能体优势需在更复杂场景中验证。
AI 中文摘要
本研究考察了工具增强的大语言模型（LLM）系统，用于支持Westermo Network Technologies AB公司夜间测试失败的根因分析（RCA）。夜间测试执行产生异构的测试数据和日志，目前从业者需要跨多个来源手动检查这些内容。我们实现了单智能体和编排多智能体两种配置的RCA工作流，两者均可访问测试元数据和日志。一项探索性工业案例研究使用了两个真实失败场景。六名从业者通过调查和焦点小组评估了场景报告，并从120次重复执行中收集了操作测量数据。评估涵盖了从业者感知的正确性、推理质量、修复现实性、清晰性、有用性和信任度，以及成本、持续时间和一致性。两种配置在两种场景中均未显示出一致的从业者感知质量优势。单智能体系统生成报告更快且成本更低，使其在此背景下成为更实用的基线。智能体架构的潜在优势需要在更复杂的场景中进一步评估。
英文摘要
This study examines tool-augmented Large Language Model (LLM) systems for supporting Root Cause Analysis (RCA) of nightly test failures at Westermo Network Technologies AB. Nightly test executions produce heterogeneous test data and logs that practitioners currently inspect manually across multiple sources. We implemented an RCA workflow in single-agent and orchestrated multi-agent configurations, both with access to test metadata and logs. An exploratory industrial case study used two real failure scenarios. Six practitioners evaluated the scenario reports through a survey and focus group, and operational measurements were collected from 120 repeated executions. The evaluation covered practitioner-perceived correctness, reasoning quality, fix realism, clarity, usefulness, and trust, as well as cost, duration, and consistency. Neither configuration showed a consistent practitioner-perceived quality advantage across the two scenarios. The single agent system generated reports faster and at lower cost, making it the more practical baseline in this context. The potential benefits of agent architectures require further evaluation in more complex scenarios.
Comments16 pages, 4 figures, accepted to PROFES 2026, 30 Nov to 2 Dec 2026, Karlskrona, Sweden
